Feature-to-Feature Regression for a Two-Step Conditional Independence Test

Qinyi Zhang, Sarah Filippi, Seth Flaxman, Dino Sejdinovic
Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, PMLR R15:381-390, 2017.

Abstract

The algorithms for causal discovery and more broadly for learning the structure of graphical models require well calibrated and consistent conditional independence (CI) tests. We revisit the CI tests which are based on two-step pro- cedures and involve regression with subsequent (unconditional) independence test (RESIT) on regression residuals and investigate the assump- tions under which these tests operate. In particu- lar, we demonstrate that when going beyond sim- ple functional relationships with additive noise, such tests can lead to an inflated number of false discoveries. We study the relationship of these tests with those based on dependence measures using reproducing kernel Hilbert spaces (RKHS) and propose an extension of RESIT which uses RKHS-valued regression. The resulting test in- herits the simple two-step testing procedure of RESIT, while giving correct Type I control and competitive power. When used as a component of the PC algorithm, the proposed test is more robust to the case where hidden variables induce a switching behaviour in the associations present in the data.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR15-zhang17b, title = {Feature-to-Feature Regression for a Two-Step Conditional Independence Test}, author = {Zhang, Qinyi and Filippi, Sarah and Flaxman, Seth and Sejdinovic, Dino}, booktitle = {Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence}, pages = {381--390}, year = {2017}, editor = {Elidan, Gal and Kersting, Kristian}, volume = {R15}, series = {Proceedings of Machine Learning Research}, month = {11--15 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r15/main/assets/zhang17b/zhang17b.pdf}, url = {https://proceedings.mlr.press/r15/zhang17b.html}, abstract = {The algorithms for causal discovery and more broadly for learning the structure of graphical models require well calibrated and consistent conditional independence (CI) tests. We revisit the CI tests which are based on two-step pro- cedures and involve regression with subsequent (unconditional) independence test (RESIT) on regression residuals and investigate the assump- tions under which these tests operate. In particu- lar, we demonstrate that when going beyond sim- ple functional relationships with additive noise, such tests can lead to an inflated number of false discoveries. We study the relationship of these tests with those based on dependence measures using reproducing kernel Hilbert spaces (RKHS) and propose an extension of RESIT which uses RKHS-valued regression. The resulting test in- herits the simple two-step testing procedure of RESIT, while giving correct Type I control and competitive power. When used as a component of the PC algorithm, the proposed test is more robust to the case where hidden variables induce a switching behaviour in the associations present in the data.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Feature-to-Feature Regression for a Two-Step Conditional Independence Test %A Qinyi Zhang %A Sarah Filippi %A Seth Flaxman %A Dino Sejdinovic %B Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2017 %E Gal Elidan %E Kristian Kersting %F pmlr-vR15-zhang17b %I PMLR %P 381--390 %U https://proceedings.mlr.press/r15/zhang17b.html %V R15 %X The algorithms for causal discovery and more broadly for learning the structure of graphical models require well calibrated and consistent conditional independence (CI) tests. We revisit the CI tests which are based on two-step pro- cedures and involve regression with subsequent (unconditional) independence test (RESIT) on regression residuals and investigate the assump- tions under which these tests operate. In particu- lar, we demonstrate that when going beyond sim- ple functional relationships with additive noise, such tests can lead to an inflated number of false discoveries. We study the relationship of these tests with those based on dependence measures using reproducing kernel Hilbert spaces (RKHS) and propose an extension of RESIT which uses RKHS-valued regression. The resulting test in- herits the simple two-step testing procedure of RESIT, while giving correct Type I control and competitive power. When used as a component of the PC algorithm, the proposed test is more robust to the case where hidden variables induce a switching behaviour in the associations present in the data. %Z Reissued by PMLR on 04 October 2026.
APA
Zhang, Q., Filippi, S., Flaxman, S. & Sejdinovic, D.. (2017). Feature-to-Feature Regression for a Two-Step Conditional Independence Test. Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R15:381-390 Available from https://proceedings.mlr.press/r15/zhang17b.html. Reissued by PMLR on 04 October 2026.

Related Material