Venn predictors for wellcalibrated probability estimation trees
[edit]
Proceedings of the Seventh Workshop on Conformal and Probabilistic Prediction and Applications, PMLR 91:314, 2018.
Abstract
Successful use of probabilistic classification requires wellcalibrated probability estimates, i.e., the predicted class probabilities must correspond to the true probabilities. The standard solution is to employ an additional step, transforming the outputs from a classifier into probability estimates. In this paper, Venn predictors are compared to Platt scaling and isotonic regression, for the purpose of producing wellcalibrated probabilistic predictions from decision trees. The empirical investigation, using 22 publicly available data sets, showed that the probability estimates from the Venn predictor were extremely wellcalibrated. In fact, in a direct comparison using the accepted reliability metric, the Venn predictor estimates were the most exact on every data set.
Related Material


