Enhancing Stability in Rule-Based Post-Hoc Explanations

Iain Smith, Osmar Zaiane
Proceedings of the The 39th Canadian Conference on Artificial Intelligence, PMLR 318:946-953, 2026.

Abstract

To trust an explanation, it must also stay the same — or at least be similar — when repeated. In much of the existing work this variance, called instability, is caused by random perturbations to the sample being explained. But this is a limited view, so in this work we study stability metrics when the data used to produce perturbations are unstable. We assess powerful explainers which use rules, where explanation instability stemming from training data becomes more apparent, and we theorize why multivariate normal distributions, producing correlated perturbed training data (+P) improve stability and fidelity in our setting. We also balance classes in the training data (+B) to further improve stability, along with exploring the potential of clustering (+C) for locality improvements to explanations. By providing both theoretical reasoning for the improvements and experiments on seven diverse datasets, with two different black-box architectures we found that the rule-based method we employed, BARBE, sharply increased in stability when trained with our modified process. BARBE+PB further exceeded the performance of other methods that improve stability like S-LIME and LORE. The final codes are available as a package on GitHub at https://github.com/IainNBSmith/Stable-BARBE.

Cite this Paper

BibTeX
@InProceedings{pmlr-v318-smith26a, title = {Enhancing Stability in Rule-Based Post-Hoc Explanations}, author = {Smith, Iain and Zaiane, Osmar}, booktitle = {Proceedings of the The 39th Canadian Conference on Artificial Intelligence}, pages = {946--953}, year = {2026}, editor = {Bouzar-Benlabiod, Lydia and Leung, Carson}, volume = {318}, series = {Proceedings of Machine Learning Research}, month = {25--29 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v318/main/assets/smith26a/smith26a.pdf}, url = {https://proceedings.mlr.press/v318/smith26a.html}, abstract = {To trust an explanation, it must also stay the same — or at least be similar — when repeated. In much of the existing work this variance, called instability, is caused by random perturbations to the sample being explained. But this is a limited view, so in this work we study stability metrics when the data used to produce perturbations are unstable. We assess powerful explainers which use rules, where explanation instability stemming from training data becomes more apparent, and we theorize why multivariate normal distributions, producing correlated perturbed training data (+P) improve stability and fidelity in our setting. We also balance classes in the training data (+B) to further improve stability, along with exploring the potential of clustering (+C) for locality improvements to explanations. By providing both theoretical reasoning for the improvements and experiments on seven diverse datasets, with two different black-box architectures we found that the rule-based method we employed, BARBE, sharply increased in stability when trained with our modified process. BARBE+PB further exceeded the performance of other methods that improve stability like S-LIME and LORE. The final codes are available as a package on GitHub at https://github.com/IainNBSmith/Stable-BARBE.} }
Endnote
%0 Conference Paper %T Enhancing Stability in Rule-Based Post-Hoc Explanations %A Iain Smith %A Osmar Zaiane %B Proceedings of the The 39th Canadian Conference on Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Lydia Bouzar-Benlabiod %E Carson Leung %F pmlr-v318-smith26a %I PMLR %P 946--953 %U https://proceedings.mlr.press/v318/smith26a.html %V 318 %X To trust an explanation, it must also stay the same — or at least be similar — when repeated. In much of the existing work this variance, called instability, is caused by random perturbations to the sample being explained. But this is a limited view, so in this work we study stability metrics when the data used to produce perturbations are unstable. We assess powerful explainers which use rules, where explanation instability stemming from training data becomes more apparent, and we theorize why multivariate normal distributions, producing correlated perturbed training data (+P) improve stability and fidelity in our setting. We also balance classes in the training data (+B) to further improve stability, along with exploring the potential of clustering (+C) for locality improvements to explanations. By providing both theoretical reasoning for the improvements and experiments on seven diverse datasets, with two different black-box architectures we found that the rule-based method we employed, BARBE, sharply increased in stability when trained with our modified process. BARBE+PB further exceeded the performance of other methods that improve stability like S-LIME and LORE. The final codes are available as a package on GitHub at https://github.com/IainNBSmith/Stable-BARBE.
APA
Smith, I. & Zaiane, O.. (2026). Enhancing Stability in Rule-Based Post-Hoc Explanations. Proceedings of the The 39th Canadian Conference on Artificial Intelligence, in Proceedings of Machine Learning Research 318:946-953 Available from https://proceedings.mlr.press/v318/smith26a.html.

Related Material