Feature Attribution for Deep Learning Models through Total Variance Decomposition

Yinzhu Jin, Shen Zhu, Tom Fletcher
Proceedings of The 8th International Conference on Medical Imaging with Deep Learning, PMLR 301:695-715, 2026.

Abstract

This paper introduces a new approach to feature attribution for deep learning models, quantifying the importance of specific features in model decisions. By decomposing the total variance of model decisions into explained and unexplained fractions, conditioned on the target feature, we define the feature attribution score as the proportion of explained variance. This method offers a solid statistical foundation and normalized quantitative results. When ample data is available, we compute the score directly from test data. For scarce data, we use constrained sampling with generative diffusion models to represent the conditional distribution at a given feature value. We demonstrate the method’s effectiveness on both a synthetic image dataset with known ground truth and OASIS-3 brain MRIs.

Cite this Paper

BibTeX
@InProceedings{pmlr-v301-jin26a, title = {Feature Attribution for Deep Learning Models through Total Variance Decomposition}, author = {Jin, Yinzhu and Zhu, Shen and Fletcher, Tom}, booktitle = {Proceedings of The 8th International Conference on Medical Imaging with Deep Learning}, pages = {695--715}, year = {2026}, editor = {Tasdizen, Tolga and Elhabian, Shireen and Summers, Ronald and Chen, Chen and Koch, Lisa and Zhuang, Yan}, volume = {301}, series = {Proceedings of Machine Learning Research}, month = {09--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v301/main/assets/jin26a/jin26a.pdf}, url = {https://proceedings.mlr.press/v301/jin26a.html}, abstract = {This paper introduces a new approach to feature attribution for deep learning models, quantifying the importance of specific features in model decisions. By decomposing the total variance of model decisions into explained and unexplained fractions, conditioned on the target feature, we define the feature attribution score as the proportion of explained variance. This method offers a solid statistical foundation and normalized quantitative results. When ample data is available, we compute the score directly from test data. For scarce data, we use constrained sampling with generative diffusion models to represent the conditional distribution at a given feature value. We demonstrate the method’s effectiveness on both a synthetic image dataset with known ground truth and OASIS-3 brain MRIs.} }
Endnote
%0 Conference Paper %T Feature Attribution for Deep Learning Models through Total Variance Decomposition %A Yinzhu Jin %A Shen Zhu %A Tom Fletcher %B Proceedings of The 8th International Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2026 %E Tolga Tasdizen %E Shireen Elhabian %E Ronald Summers %E Chen Chen %E Lisa Koch %E Yan Zhuang %F pmlr-v301-jin26a %I PMLR %P 695--715 %U https://proceedings.mlr.press/v301/jin26a.html %V 301 %X This paper introduces a new approach to feature attribution for deep learning models, quantifying the importance of specific features in model decisions. By decomposing the total variance of model decisions into explained and unexplained fractions, conditioned on the target feature, we define the feature attribution score as the proportion of explained variance. This method offers a solid statistical foundation and normalized quantitative results. When ample data is available, we compute the score directly from test data. For scarce data, we use constrained sampling with generative diffusion models to represent the conditional distribution at a given feature value. We demonstrate the method’s effectiveness on both a synthetic image dataset with known ground truth and OASIS-3 brain MRIs.
APA
Jin, Y., Zhu, S. & Fletcher, T.. (2026). Feature Attribution for Deep Learning Models through Total Variance Decomposition. Proceedings of The 8th International Conference on Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 301:695-715 Available from https://proceedings.mlr.press/v301/jin26a.html.

Related Material