Assessing Explanation Quality by Venn Prediction

Amr Alkhatib, Henrik Boström, Ulf Johansson
Proceedings of the Eleventh Symposium on Conformal and Probabilistic Prediction with Applications, PMLR 179:42-54, 2022.

Abstract

Rules output by explainable machine learning techniques naturally come with a degree of uncertainty, as the complex functionality of the underlying black-box model often can be difficult to approximate by a single, interpretable rule. However, the uncertainty of these approximations is not properly quantified by current explanatory techniques. The use of Venn prediction is here proposed and investigated as a means to quantify the uncertainty of the explanations and thereby also allow for competing explanation techniques to be evaluated with respect to their relative uncertainty. A number of metrics of rule explanation quality based on uncertainty are proposed and discussed, including metrics that capture the tendency of the explanations to predict the correct outcome of a black-box model on new instances, how informative (tight) the produced intervals are, and how certain a rule is when predicting one class. An empirical investigation is presented, in which explanations produced by the state-of-the-art technique Anchors are compared to explanatory rules obtained from association rule mining. The results suggest that the association rule mining approach may provide explanations with less uncertainty towards the correct label, as predicted by the black-box model, compared to Anchors. The results also show that the explanatory rules obtained through association rule mining result in tighter intervals and are closer to either one or zero compared to Anchors, i.e., they are more certain towards a specific class label.

Cite this Paper

BibTeX
@InProceedings{pmlr-v179-alkhatib22a, title = {Assessing Explanation Quality by Venn Prediction}, author = {Alkhatib, Amr and Bostr\"{o}m, Henrik and Johansson, Ulf}, booktitle = {Proceedings of the Eleventh Symposium on Conformal and Probabilistic Prediction with Applications}, pages = {42--54}, year = {2022}, editor = {Johansson, Ulf and Boström, Henrik and An Nguyen, Khuong and Luo, Zhiyuan and Carlsson, Lars}, volume = {179}, series = {Proceedings of Machine Learning Research}, month = {24--26 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v179/alkhatib22a/alkhatib22a.pdf}, url = {https://proceedings.mlr.press/v179/alkhatib22a.html}, abstract = { Rules output by explainable machine learning techniques naturally come with a degree of uncertainty, as the complex functionality of the underlying black-box model often can be difficult to approximate by a single, interpretable rule. However, the uncertainty of these approximations is not properly quantified by current explanatory techniques. The use of Venn prediction is here proposed and investigated as a means to quantify the uncertainty of the explanations and thereby also allow for competing explanation techniques to be evaluated with respect to their relative uncertainty. A number of metrics of rule explanation quality based on uncertainty are proposed and discussed, including metrics that capture the tendency of the explanations to predict the correct outcome of a black-box model on new instances, how informative (tight) the produced intervals are, and how certain a rule is when predicting one class. An empirical investigation is presented, in which explanations produced by the state-of-the-art technique Anchors are compared to explanatory rules obtained from association rule mining. The results suggest that the association rule mining approach may provide explanations with less uncertainty towards the correct label, as predicted by the black-box model, compared to Anchors. The results also show that the explanatory rules obtained through association rule mining result in tighter intervals and are closer to either one or zero compared to Anchors, i.e., they are more certain towards a specific class label. } }
Endnote
%0 Conference Paper %T Assessing Explanation Quality by Venn Prediction %A Amr Alkhatib %A Henrik Boström %A Ulf Johansson %B Proceedings of the Eleventh Symposium on Conformal and Probabilistic Prediction with Applications %C Proceedings of Machine Learning Research %D 2022 %E Ulf Johansson %E Henrik Boström %E Khuong An Nguyen %E Zhiyuan Luo %E Lars Carlsson %F pmlr-v179-alkhatib22a %I PMLR %P 42--54 %U https://proceedings.mlr.press/v179/alkhatib22a.html %V 179 %X Rules output by explainable machine learning techniques naturally come with a degree of uncertainty, as the complex functionality of the underlying black-box model often can be difficult to approximate by a single, interpretable rule. However, the uncertainty of these approximations is not properly quantified by current explanatory techniques. The use of Venn prediction is here proposed and investigated as a means to quantify the uncertainty of the explanations and thereby also allow for competing explanation techniques to be evaluated with respect to their relative uncertainty. A number of metrics of rule explanation quality based on uncertainty are proposed and discussed, including metrics that capture the tendency of the explanations to predict the correct outcome of a black-box model on new instances, how informative (tight) the produced intervals are, and how certain a rule is when predicting one class. An empirical investigation is presented, in which explanations produced by the state-of-the-art technique Anchors are compared to explanatory rules obtained from association rule mining. The results suggest that the association rule mining approach may provide explanations with less uncertainty towards the correct label, as predicted by the black-box model, compared to Anchors. The results also show that the explanatory rules obtained through association rule mining result in tighter intervals and are closer to either one or zero compared to Anchors, i.e., they are more certain towards a specific class label.
APA
Alkhatib, A., Boström, H. & Johansson, U.. (2022). Assessing Explanation Quality by Venn Prediction. Proceedings of the Eleventh Symposium on Conformal and Probabilistic Prediction with Applications, in Proceedings of Machine Learning Research 179:42-54 Available from https://proceedings.mlr.press/v179/alkhatib22a.html.

Related Material