Conformal predictor combination using Neyman–Pearson Lemma

Paolo Toccaceli
Proceedings of the Eighth Symposium on Conformal and Probabilistic Prediction and Applications, PMLR 105:66-88, 2019.

Abstract

The problem of how to combine advantageously Conformal Predictors (CP) has attracted the interest of many researchers in recent years. The challenge is to retain validity, while improving efficiency. In this article a very generic method is proposed which takes advantage of a well-established result in Classical Statistical Hypothesis Testing, the Neyman–Pearson Lemma, to combine CP with maximum efficiency. The merits and the limits of the method are explored on synthetic data sets under different levels of correlation between NonConformity Measures (NCM). CP Combination via Neyman–Pearson Lemma generally outperforms other combination methods when an accurate and robust density ratio estimation method, such as the V-Matrix method, is used.

Cite this Paper

BibTeX
@InProceedings{pmlr-v105-toccaceli19a, title = {Conformal predictor combination using {N}eyman–{P}earson Lemma}, author = {Toccaceli, Paolo}, booktitle = {Proceedings of the Eighth Symposium on Conformal and Probabilistic Prediction and Applications}, pages = {66--88}, year = {2019}, editor = {Gammerman, Alex and Vovk, Vladimir and Luo, Zhiyuan and Smirnov, Evgueni}, volume = {105}, series = {Proceedings of Machine Learning Research}, month = {09--11 Sep}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v105/toccaceli19a/toccaceli19a.pdf}, url = {https://proceedings.mlr.press/v105/toccaceli19a.html}, abstract = {The problem of how to combine advantageously Conformal Predictors (CP) has attracted the interest of many researchers in recent years. The challenge is to retain validity, while improving efficiency. In this article a very generic method is proposed which takes advantage of a well-established result in Classical Statistical Hypothesis Testing, the Neyman–Pearson Lemma, to combine CP with maximum efficiency. The merits and the limits of the method are explored on synthetic data sets under different levels of correlation between NonConformity Measures (NCM). CP Combination via Neyman–Pearson Lemma generally outperforms other combination methods when an accurate and robust density ratio estimation method, such as the V-Matrix method, is used.} }
Endnote
%0 Conference Paper %T Conformal predictor combination using Neyman–Pearson Lemma %A Paolo Toccaceli %B Proceedings of the Eighth Symposium on Conformal and Probabilistic Prediction and Applications %C Proceedings of Machine Learning Research %D 2019 %E Alex Gammerman %E Vladimir Vovk %E Zhiyuan Luo %E Evgueni Smirnov %F pmlr-v105-toccaceli19a %I PMLR %P 66--88 %U https://proceedings.mlr.press/v105/toccaceli19a.html %V 105 %X The problem of how to combine advantageously Conformal Predictors (CP) has attracted the interest of many researchers in recent years. The challenge is to retain validity, while improving efficiency. In this article a very generic method is proposed which takes advantage of a well-established result in Classical Statistical Hypothesis Testing, the Neyman–Pearson Lemma, to combine CP with maximum efficiency. The merits and the limits of the method are explored on synthetic data sets under different levels of correlation between NonConformity Measures (NCM). CP Combination via Neyman–Pearson Lemma generally outperforms other combination methods when an accurate and robust density ratio estimation method, such as the V-Matrix method, is used.
APA
Toccaceli, P.. (2019). Conformal predictor combination using Neyman–Pearson Lemma. Proceedings of the Eighth Symposium on Conformal and Probabilistic Prediction and Applications, in Proceedings of Machine Learning Research 105:66-88 Available from https://proceedings.mlr.press/v105/toccaceli19a.html.

Related Material