Partial Information Decomposition for Data Interpretability and Feature Selection

Charles Westphal, Stephen Hailes, Mirco Musolesi
Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, PMLR 258:1873-1881, 2025.

Abstract

In this paper, we introduce Partial Information Decomposition of Features (PIDF), a new paradigm for simultaneous data interpretability and feature selection. Contrary to traditional methods that assign a single importance value, our approach is based on three metrics per feature: the mutual information shared with the target variable, the feature’s contribution to synergistic information, and the amount of this information that is redundant. In particular, we develop a novel procedure based on these three metrics, which reveals not only how features are correlated with the target but also the additional and overlapping information provided by considering them in combination with other features. We extensively evaluate PIDF using both synthetic and real-world data, demonstrating its potential applications and effectiveness, by considering case studies from genetics and neuroscience.

Cite this Paper

BibTeX
@InProceedings{pmlr-v258-westphal25a, title = {Partial Information Decomposition for Data Interpretability and Feature Selection}, author = {Westphal, Charles and Hailes, Stephen and Musolesi, Mirco}, booktitle = {Proceedings of The 28th International Conference on Artificial Intelligence and Statistics}, pages = {1873--1881}, year = {2025}, editor = {Li, Yingzhen and Mandt, Stephan and Agrawal, Shipra and Khan, Emtiyaz}, volume = {258}, series = {Proceedings of Machine Learning Research}, month = {03--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v258/main/assets/westphal25a/westphal25a.pdf}, url = {https://proceedings.mlr.press/v258/westphal25a.html}, abstract = {In this paper, we introduce Partial Information Decomposition of Features (PIDF), a new paradigm for simultaneous data interpretability and feature selection. Contrary to traditional methods that assign a single importance value, our approach is based on three metrics per feature: the mutual information shared with the target variable, the feature’s contribution to synergistic information, and the amount of this information that is redundant. In particular, we develop a novel procedure based on these three metrics, which reveals not only how features are correlated with the target but also the additional and overlapping information provided by considering them in combination with other features. We extensively evaluate PIDF using both synthetic and real-world data, demonstrating its potential applications and effectiveness, by considering case studies from genetics and neuroscience.} }
Endnote
%0 Conference Paper %T Partial Information Decomposition for Data Interpretability and Feature Selection %A Charles Westphal %A Stephen Hailes %A Mirco Musolesi %B Proceedings of The 28th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2025 %E Yingzhen Li %E Stephan Mandt %E Shipra Agrawal %E Emtiyaz Khan %F pmlr-v258-westphal25a %I PMLR %P 1873--1881 %U https://proceedings.mlr.press/v258/westphal25a.html %V 258 %X In this paper, we introduce Partial Information Decomposition of Features (PIDF), a new paradigm for simultaneous data interpretability and feature selection. Contrary to traditional methods that assign a single importance value, our approach is based on three metrics per feature: the mutual information shared with the target variable, the feature’s contribution to synergistic information, and the amount of this information that is redundant. In particular, we develop a novel procedure based on these three metrics, which reveals not only how features are correlated with the target but also the additional and overlapping information provided by considering them in combination with other features. We extensively evaluate PIDF using both synthetic and real-world data, demonstrating its potential applications and effectiveness, by considering case studies from genetics and neuroscience.
APA
Westphal, C., Hailes, S. & Musolesi, M.. (2025). Partial Information Decomposition for Data Interpretability and Feature Selection. Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 258:1873-1881 Available from https://proceedings.mlr.press/v258/westphal25a.html.

Related Material