Synergy conformal prediction

Niharika Gauraha, Ola Spjuth
Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications, PMLR 152:91-110, 2021.

Abstract

Conformal prediction is a machine learning methodology that produces valid prediction regions. Ensembles of conformal predictors have been proposed to improve the informational efficiency of inductive conformal predictors by combining p-values, however, the validity of such methods has been an open problem. We introduce synergy conformal prediction which is an ensemble method that combines monotonic conformity scores, and is capable of producing valid prediction intervals. We study the applicability in three scenarios; where data is partitioned, where an ensemble of different machine learning methods is used, and where data is unpartitioned. We evaluate the method on 10 data sets and show that the synergy conformal predictor produces valid prediction intervals that on partitioned data performs well compared to the most efficient model trained on individual partitions, making it a viable approach for federated settings when data cannot be pooled. We also show that our method has advantages over current ensembles of conformal predictors by producing valid and efficient results on unpartitioned data, and that it is less computationally demanding.

Cite this Paper

BibTeX
@InProceedings{pmlr-v152-gauraha21a, title = {Synergy conformal prediction}, author = {Gauraha, Niharika and Spjuth, Ola}, booktitle = {Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications}, pages = {91--110}, year = {2021}, editor = {Carlsson, Lars and Luo, Zhiyuan and Cherubin, Giovanni and An Nguyen, Khuong}, volume = {152}, series = {Proceedings of Machine Learning Research}, month = {08--10 Sep}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v152/gauraha21a/gauraha21a.pdf}, url = {https://proceedings.mlr.press/v152/gauraha21a.html}, abstract = {Conformal prediction is a machine learning methodology that produces valid prediction regions. Ensembles of conformal predictors have been proposed to improve the informational efficiency of inductive conformal predictors by combining p-values, however, the validity of such methods has been an open problem. We introduce synergy conformal prediction which is an ensemble method that combines monotonic conformity scores, and is capable of producing valid prediction intervals. We study the applicability in three scenarios; where data is partitioned, where an ensemble of different machine learning methods is used, and where data is unpartitioned. We evaluate the method on 10 data sets and show that the synergy conformal predictor produces valid prediction intervals that on partitioned data performs well compared to the most efficient model trained on individual partitions, making it a viable approach for federated settings when data cannot be pooled. We also show that our method has advantages over current ensembles of conformal predictors by producing valid and efficient results on unpartitioned data, and that it is less computationally demanding.} }
Endnote
%0 Conference Paper %T Synergy conformal prediction %A Niharika Gauraha %A Ola Spjuth %B Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications %C Proceedings of Machine Learning Research %D 2021 %E Lars Carlsson %E Zhiyuan Luo %E Giovanni Cherubin %E Khuong An Nguyen %F pmlr-v152-gauraha21a %I PMLR %P 91--110 %U https://proceedings.mlr.press/v152/gauraha21a.html %V 152 %X Conformal prediction is a machine learning methodology that produces valid prediction regions. Ensembles of conformal predictors have been proposed to improve the informational efficiency of inductive conformal predictors by combining p-values, however, the validity of such methods has been an open problem. We introduce synergy conformal prediction which is an ensemble method that combines monotonic conformity scores, and is capable of producing valid prediction intervals. We study the applicability in three scenarios; where data is partitioned, where an ensemble of different machine learning methods is used, and where data is unpartitioned. We evaluate the method on 10 data sets and show that the synergy conformal predictor produces valid prediction intervals that on partitioned data performs well compared to the most efficient model trained on individual partitions, making it a viable approach for federated settings when data cannot be pooled. We also show that our method has advantages over current ensembles of conformal predictors by producing valid and efficient results on unpartitioned data, and that it is less computationally demanding.
APA
Gauraha, N. & Spjuth, O.. (2021). Synergy conformal prediction. Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications, in Proceedings of Machine Learning Research 152:91-110 Available from https://proceedings.mlr.press/v152/gauraha21a.html.

Related Material