Algorithmic causal structure emerging through compression

Liang Wendong, Simon Buchholz, Bernhard Schölkopf
Proceedings of the Fourth Conference on Causal Learning and Reasoning, PMLR 275:201-242, 2025.

Abstract

We explore the relationship between causality, symmetry, and compression. We build on and generalize the known connection between learning and compression to a setting where causal models are not identifiable. We propose a framework where causality emerges as a consequence of compressing data across multiple environments. We define algorithmic causality as an alternative definition of causality when traditional assumptions for causal identifiability do not hold. We demonstrate how algorithmic causal and symmetric structures can emerge from minimizing upper bounds on Kolmogorov complexity, without knowledge of intervention targets. We hypothesize that these insights may also provide a novel perspective on the emergence of causality in machine learning models, such as large language models, where causal relationships may not be explicitly identifiable.

Cite this Paper

BibTeX
@InProceedings{pmlr-v275-wendong25a, title = {Algorithmic causal structure emerging through compression}, author = {Wendong, Liang and Buchholz, Simon and Sch\"{o}lkopf, Bernhard}, booktitle = {Proceedings of the Fourth Conference on Causal Learning and Reasoning}, pages = {201--242}, year = {2025}, editor = {Huang, Biwei and Drton, Mathias}, volume = {275}, series = {Proceedings of Machine Learning Research}, month = {07--09 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v275/main/assets/wendong25a/wendong25a.pdf}, url = {https://proceedings.mlr.press/v275/wendong25a.html}, abstract = {We explore the relationship between causality, symmetry, and compression. We build on and generalize the known connection between learning and compression to a setting where causal models are not identifiable. We propose a framework where causality emerges as a consequence of compressing data across multiple environments. We define algorithmic causality as an alternative definition of causality when traditional assumptions for causal identifiability do not hold. We demonstrate how algorithmic causal and symmetric structures can emerge from minimizing upper bounds on Kolmogorov complexity, without knowledge of intervention targets. We hypothesize that these insights may also provide a novel perspective on the emergence of causality in machine learning models, such as large language models, where causal relationships may not be explicitly identifiable.} }
Endnote
%0 Conference Paper %T Algorithmic causal structure emerging through compression %A Liang Wendong %A Simon Buchholz %A Bernhard Schölkopf %B Proceedings of the Fourth Conference on Causal Learning and Reasoning %C Proceedings of Machine Learning Research %D 2025 %E Biwei Huang %E Mathias Drton %F pmlr-v275-wendong25a %I PMLR %P 201--242 %U https://proceedings.mlr.press/v275/wendong25a.html %V 275 %X We explore the relationship between causality, symmetry, and compression. We build on and generalize the known connection between learning and compression to a setting where causal models are not identifiable. We propose a framework where causality emerges as a consequence of compressing data across multiple environments. We define algorithmic causality as an alternative definition of causality when traditional assumptions for causal identifiability do not hold. We demonstrate how algorithmic causal and symmetric structures can emerge from minimizing upper bounds on Kolmogorov complexity, without knowledge of intervention targets. We hypothesize that these insights may also provide a novel perspective on the emergence of causality in machine learning models, such as large language models, where causal relationships may not be explicitly identifiable.
APA
Wendong, L., Buchholz, S. & Schölkopf, B.. (2025). Algorithmic causal structure emerging through compression. Proceedings of the Fourth Conference on Causal Learning and Reasoning, in Proceedings of Machine Learning Research 275:201-242 Available from https://proceedings.mlr.press/v275/wendong25a.html.

Related Material