Hide&Seek: Learning to Explain in an End-to-End Differentiable Network

Tal Ellinson, Hadi Mohasel Afshar, Sally Cripps
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:27939-27965, 2026.

Abstract

Instance-wise feature selection is a valuable tool for interpreting labeled data and the predictions of black-box models. In contrast to global feature selection techniques, instance-wise methods dynamically identify important features for each instance. A growing number of methods learn a selector, which identifies important features, and a predictor, which uses these to make predictions. However, these pioneering methods face challenges including information leakage and lack of differentiability, which can slow training. In this paper, we present Hide&Seek, an end-to-end differentiable model for instance-wise feature selection. We jointly learn feature selection and prediction under a single objective without information leakage. Hide&Seek outperforms existing state-of-the-art models across a range of experiments and is fast to train. We achieve this by reformulating feature removal as a differentiable operation where instead of discretely removing features, we replace a proportion of each feature. Training is further stabilized via a parsimony-weight annealing framework.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-ellinson26a, title = {Hide&Seek: Learning to Explain in an End-to-End Differentiable Network}, author = {Ellinson, Tal and Afshar, Hadi Mohasel and Cripps, Sally}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {27939--27965}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/ellinson26a/ellinson26a.pdf}, url = {https://proceedings.mlr.press/v306/ellinson26a.html}, abstract = {Instance-wise feature selection is a valuable tool for interpreting labeled data and the predictions of black-box models. In contrast to global feature selection techniques, instance-wise methods dynamically identify important features for each instance. A growing number of methods learn a selector, which identifies important features, and a predictor, which uses these to make predictions. However, these pioneering methods face challenges including information leakage and lack of differentiability, which can slow training. In this paper, we present Hide&Seek, an end-to-end differentiable model for instance-wise feature selection. We jointly learn feature selection and prediction under a single objective without information leakage. Hide&Seek outperforms existing state-of-the-art models across a range of experiments and is fast to train. We achieve this by reformulating feature removal as a differentiable operation where instead of discretely removing features, we replace a proportion of each feature. Training is further stabilized via a parsimony-weight annealing framework.} }
Endnote
%0 Conference Paper %T Hide&Seek: Learning to Explain in an End-to-End Differentiable Network %A Tal Ellinson %A Hadi Mohasel Afshar %A Sally Cripps %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-ellinson26a %I PMLR %P 27939--27965 %U https://proceedings.mlr.press/v306/ellinson26a.html %V 306 %X Instance-wise feature selection is a valuable tool for interpreting labeled data and the predictions of black-box models. In contrast to global feature selection techniques, instance-wise methods dynamically identify important features for each instance. A growing number of methods learn a selector, which identifies important features, and a predictor, which uses these to make predictions. However, these pioneering methods face challenges including information leakage and lack of differentiability, which can slow training. In this paper, we present Hide&Seek, an end-to-end differentiable model for instance-wise feature selection. We jointly learn feature selection and prediction under a single objective without information leakage. Hide&Seek outperforms existing state-of-the-art models across a range of experiments and is fast to train. We achieve this by reformulating feature removal as a differentiable operation where instead of discretely removing features, we replace a proportion of each feature. Training is further stabilized via a parsimony-weight annealing framework.
APA
Ellinson, T., Afshar, H.M. & Cripps, S.. (2026). Hide&Seek: Learning to Explain in an End-to-End Differentiable Network. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:27939-27965 Available from https://proceedings.mlr.press/v306/ellinson26a.html.

Related Material