Beyond the Predicted Class: Calibrated Explanations for Real Multiclass Decisions*

Ayham Hanna, Tuwe Löfström
Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications, PMLR 329:402-421, 2026.

Abstract

Calibrated Explanations provide uncertainty-aware feature importance and probabilistic predictions for binary classification, but existing multiclass extensions explain only the most probable class. This limitation reduces interpretability, especially when prediction confidence is low. We extend Calibrated Explanations to full multiclass support using a one-vs-rest strategy, generating calibrated probability intervals and class-specific feature importance for all classes. Experiments on four datasets show lower repeated-run variance than calibrated SHAP and LIME, while uncalibrated SHAP and LIME achieved lower variance in the robustness evaluation. A user study with 22 experts found that the proposed explanation format received the highest mean trust score and strongest overall preference ranking, while also influencing participants’ decisions.

Cite this Paper

BibTeX
@InProceedings{pmlr-v329-hanna26a, title = {Beyond the Predicted Class: Calibrated Explanations for Real Multiclass Decisions*}, author = {Hanna, Ayham and L{\"o}fstr{\"o}m, Tuwe}, booktitle = {Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications}, pages = {402--421}, year = {2026}, editor = {Ahlberg, Ernst and Johansson, Ulf and Boström, Henrik and Carlevaro, Alberto and Hallberg Szabadváry, Johan and Carlsson, Lars}, volume = {329}, series = {Proceedings of Machine Learning Research}, month = {02--04 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v329/main/assets/hanna26a/hanna26a.pdf}, url = {https://proceedings.mlr.press/v329/hanna26a.html}, abstract = {Calibrated Explanations provide uncertainty-aware feature importance and probabilistic predictions for binary classification, but existing multiclass extensions explain only the most probable class. This limitation reduces interpretability, especially when prediction confidence is low. We extend Calibrated Explanations to full multiclass support using a one-vs-rest strategy, generating calibrated probability intervals and class-specific feature importance for all classes. Experiments on four datasets show lower repeated-run variance than calibrated SHAP and LIME, while uncalibrated SHAP and LIME achieved lower variance in the robustness evaluation. A user study with 22 experts found that the proposed explanation format received the highest mean trust score and strongest overall preference ranking, while also influencing participants’ decisions.} }
Endnote
%0 Conference Paper %T Beyond the Predicted Class: Calibrated Explanations for Real Multiclass Decisions* %A Ayham Hanna %A Tuwe Löfström %B Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications %C Proceedings of Machine Learning Research %D 2026 %E Ernst Ahlberg %E Ulf Johansson %E Henrik Boström %E Alberto Carlevaro %E Johan Hallberg Szabadváry %E Lars Carlsson %F pmlr-v329-hanna26a %I PMLR %P 402--421 %U https://proceedings.mlr.press/v329/hanna26a.html %V 329 %X Calibrated Explanations provide uncertainty-aware feature importance and probabilistic predictions for binary classification, but existing multiclass extensions explain only the most probable class. This limitation reduces interpretability, especially when prediction confidence is low. We extend Calibrated Explanations to full multiclass support using a one-vs-rest strategy, generating calibrated probability intervals and class-specific feature importance for all classes. Experiments on four datasets show lower repeated-run variance than calibrated SHAP and LIME, while uncalibrated SHAP and LIME achieved lower variance in the robustness evaluation. A user study with 22 experts found that the proposed explanation format received the highest mean trust score and strongest overall preference ranking, while also influencing participants’ decisions.
APA
Hanna, A. & Löfström, T.. (2026). Beyond the Predicted Class: Calibrated Explanations for Real Multiclass Decisions*. Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications, in Proceedings of Machine Learning Research 329:402-421 Available from https://proceedings.mlr.press/v329/hanna26a.html.

Related Material