Combination of Conformal Predictors for Classification
Proceedings of the Sixth Workshop on Conformal and Probabilistic Prediction and Applications, PMLR 60:39-61, 2017.
The paper presents some possible approaches to the combination of Conformal Predictors in the binary classification case. A first class of methods is based on p-value combination techniques that have been proposed in the context of Statistical Hypothesis Testing; a second class is based on the calibration of p-values into Bayes factors. A few methods from these two classes are applied to a real-world case, namely the chemoinformatics problem of Compound Activity Prediction. Their performance is discussed, showing the different abilities to preserve of validity and improve efficiency. The experiments show that P-value combination, in particular Fisher’s method, can be advantageous when ranking compounds by strength of evidence.