Causal Conclusions that Flip Repeatedly

Kevin Kelly, Conor Mayo-Wilson
Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, PMLR R8:285-292, 2010.

Abstract

Over the past two decades, several consis- tent procedures have been designed to infer causal conclusions from observational data. We prove that if the true causal network might be an arbitrary, linear Gaussian net- work or a discrete Bayes network, then every unambiguous causal conclusion produced by a consistent method from non-experimental data is subject to reversal as the sample size increases any finite number of times. That result, called the causal flipping theorem, ex- tends prior results to the effect that causal discovery cannot be reliable on a given sam- ple size. We argue that since repeated flip- ping of causal conclusions is unavoidable in principle for consistent methods, the best possible discovery methods are consistent methods that retract their earlier conclusions no more than necessary. A series of sim- ulations of various methods across a wide range of sample sizes illustrates concretely both the theorem and the principle of com- paring methods in terms of retractions.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR8-kelly10a, title = {Causal Conclusions that Flip Repeatedly}, author = {Kelly, Kevin and Mayo-Wilson, Conor}, booktitle = {Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence}, pages = {285--292}, year = {2010}, editor = {Grünwald, Peter and Spirtes, Peter}, volume = {R8}, series = {Proceedings of Machine Learning Research}, month = {08--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r8/main/assets/kelly10a/kelly10a.pdf}, url = {https://proceedings.mlr.press/r8/kelly10a.html}, abstract = {Over the past two decades, several consis- tent procedures have been designed to infer causal conclusions from observational data. We prove that if the true causal network might be an arbitrary, linear Gaussian net- work or a discrete Bayes network, then every unambiguous causal conclusion produced by a consistent method from non-experimental data is subject to reversal as the sample size increases any finite number of times. That result, called the causal flipping theorem, ex- tends prior results to the effect that causal discovery cannot be reliable on a given sam- ple size. We argue that since repeated flip- ping of causal conclusions is unavoidable in principle for consistent methods, the best possible discovery methods are consistent methods that retract their earlier conclusions no more than necessary. A series of sim- ulations of various methods across a wide range of sample sizes illustrates concretely both the theorem and the principle of com- paring methods in terms of retractions.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Causal Conclusions that Flip Repeatedly %A Kevin Kelly %A Conor Mayo-Wilson %B Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2010 %E Peter Grünwald %E Peter Spirtes %F pmlr-vR8-kelly10a %I PMLR %P 285--292 %U https://proceedings.mlr.press/r8/kelly10a.html %V R8 %X Over the past two decades, several consis- tent procedures have been designed to infer causal conclusions from observational data. We prove that if the true causal network might be an arbitrary, linear Gaussian net- work or a discrete Bayes network, then every unambiguous causal conclusion produced by a consistent method from non-experimental data is subject to reversal as the sample size increases any finite number of times. That result, called the causal flipping theorem, ex- tends prior results to the effect that causal discovery cannot be reliable on a given sam- ple size. We argue that since repeated flip- ping of causal conclusions is unavoidable in principle for consistent methods, the best possible discovery methods are consistent methods that retract their earlier conclusions no more than necessary. A series of sim- ulations of various methods across a wide range of sample sizes illustrates concretely both the theorem and the principle of com- paring methods in terms of retractions. %Z Reissued by PMLR on 04 October 2026.
APA
Kelly, K. & Mayo-Wilson, C.. (2010). Causal Conclusions that Flip Repeatedly. Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R8:285-292 Available from https://proceedings.mlr.press/r8/kelly10a.html. Reissued by PMLR on 04 October 2026.

Related Material