Intervening to learn and compose causally disentangled representations

Alex Markham, Isaac Hirsch, Jeri A. Chang, Liam Solus, Bryon Aragam
Proceedings of the Fifth Conference on Causal Learning and Reasoning, PMLR 323:1481-1525, 2026.

Abstract

In designing generative models, it is commonly believed that in order to learn useful latent structure, we face a fundamental tension between expressivity and structure. In this paper we challenge this view by proposing a new approach to training arbitrarily expressive generative models that simultaneously learn causally disentangled concepts. This is accomplished by adding a simple \emph{context module} to an arbitrarily complex black-box model, which learns to process concept information by implicitly inverting linear representations from the model’s encoder. Inspired by the notion of intervention in a causal model, our module selectively modifies its architecture during training, allowing it to learn a compact joint model over different contexts. We show how adding this module leads to causally disentangled representations that can be composed for out-of-distribution generation on both real and simulated data. The resulting models can be trained end-to-end or fine-tuned from pre-trained models. To further validate our proposed approach, we prove a new identifiability result that extends existing work on identifying structured representations.

Cite this Paper

BibTeX
@InProceedings{pmlr-v323-markham26a, title = {Intervening to learn and compose causally disentangled representations}, author = {Markham, Alex and Hirsch, Isaac and Chang, Jeri A. and Solus, Liam and Aragam, Bryon}, booktitle = {Proceedings of the Fifth Conference on Causal Learning and Reasoning}, pages = {1481--1525}, year = {2026}, editor = {Mazaheri, Bijan and Hanson, Niels Richard}, volume = {323}, series = {Proceedings of Machine Learning Research}, month = {06--08 Apr}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v323/main/assets/markham26a/markham26a.pdf}, url = {https://proceedings.mlr.press/v323/markham26a.html}, abstract = {In designing generative models, it is commonly believed that in order to learn useful latent structure, we face a fundamental tension between expressivity and structure. In this paper we challenge this view by proposing a new approach to training arbitrarily expressive generative models that simultaneously learn causally disentangled concepts. This is accomplished by adding a simple \emph{context module} to an arbitrarily complex black-box model, which learns to process concept information by implicitly inverting linear representations from the model’s encoder. Inspired by the notion of intervention in a causal model, our module selectively modifies its architecture during training, allowing it to learn a compact joint model over different contexts. We show how adding this module leads to causally disentangled representations that can be composed for out-of-distribution generation on both real and simulated data. The resulting models can be trained end-to-end or fine-tuned from pre-trained models. To further validate our proposed approach, we prove a new identifiability result that extends existing work on identifying structured representations.} }
Endnote
%0 Conference Paper %T Intervening to learn and compose causally disentangled representations %A Alex Markham %A Isaac Hirsch %A Jeri A. Chang %A Liam Solus %A Bryon Aragam %B Proceedings of the Fifth Conference on Causal Learning and Reasoning %C Proceedings of Machine Learning Research %D 2026 %E Bijan Mazaheri %E Niels Richard Hanson %F pmlr-v323-markham26a %I PMLR %P 1481--1525 %U https://proceedings.mlr.press/v323/markham26a.html %V 323 %X In designing generative models, it is commonly believed that in order to learn useful latent structure, we face a fundamental tension between expressivity and structure. In this paper we challenge this view by proposing a new approach to training arbitrarily expressive generative models that simultaneously learn causally disentangled concepts. This is accomplished by adding a simple \emph{context module} to an arbitrarily complex black-box model, which learns to process concept information by implicitly inverting linear representations from the model’s encoder. Inspired by the notion of intervention in a causal model, our module selectively modifies its architecture during training, allowing it to learn a compact joint model over different contexts. We show how adding this module leads to causally disentangled representations that can be composed for out-of-distribution generation on both real and simulated data. The resulting models can be trained end-to-end or fine-tuned from pre-trained models. To further validate our proposed approach, we prove a new identifiability result that extends existing work on identifying structured representations.
APA
Markham, A., Hirsch, I., Chang, J.A., Solus, L. & Aragam, B.. (2026). Intervening to learn and compose causally disentangled representations. Proceedings of the Fifth Conference on Causal Learning and Reasoning, in Proceedings of Machine Learning Research 323:1481-1525 Available from https://proceedings.mlr.press/v323/markham26a.html.

Related Material