Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA

Sebastien Lachapelle, Pau Rodriguez, Yash Sharma, Katie E Everett, Rémi LE PRIOL, Alexandre Lacoste, Simon Lacoste-Julien
Proceedings of the First Conference on Causal Learning and Reasoning, PMLR 177:428-484, 2022.

Abstract

This work introduces a novel principle we call disentanglement via mechanism sparsity regularization, which can be applied when the latent factors of interest depend sparsely on past latent factors and/or observed auxiliary variables. We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors and the sparse causal graphical model that relates them. We develop a rigorous identifiability theory, building on recent nonlinear independent component analysis (ICA) results, that formalizes this principle and shows how the latent variables can be recovered up to permutation if one regularizes the latent mechanisms to be sparse and if some graph connectivity criterion is satisfied by the data generating process. As a special case of our framework, we show how one can leverage unknown-target interventions on the latent factors to disentangle them, thereby drawing further connections between ICA and causality. We propose a VAE-based method in which the latent mechanisms are learned and regularized via binary masks, and validate our theory by showing it learns disentangled representations in simulations.

Cite this Paper

BibTeX
@InProceedings{pmlr-v177-lachapelle22a, title = {Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear {ICA}}, author = {Lachapelle, Sebastien and Rodriguez, Pau and Sharma, Yash and Everett, Katie E and PRIOL, R{\'e}mi LE and Lacoste, Alexandre and Lacoste-Julien, Simon}, booktitle = {Proceedings of the First Conference on Causal Learning and Reasoning}, pages = {428--484}, year = {2022}, editor = {Schölkopf, Bernhard and Uhler, Caroline and Zhang, Kun}, volume = {177}, series = {Proceedings of Machine Learning Research}, month = {11--13 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v177/lachapelle22a/lachapelle22a.pdf}, url = {https://proceedings.mlr.press/v177/lachapelle22a.html}, abstract = {This work introduces a novel principle we call disentanglement via mechanism sparsity regularization, which can be applied when the latent factors of interest depend sparsely on past latent factors and/or observed auxiliary variables. We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors and the sparse causal graphical model that relates them. We develop a rigorous identifiability theory, building on recent nonlinear independent component analysis (ICA) results, that formalizes this principle and shows how the latent variables can be recovered up to permutation if one regularizes the latent mechanisms to be sparse and if some graph connectivity criterion is satisfied by the data generating process. As a special case of our framework, we show how one can leverage unknown-target interventions on the latent factors to disentangle them, thereby drawing further connections between ICA and causality. We propose a VAE-based method in which the latent mechanisms are learned and regularized via binary masks, and validate our theory by showing it learns disentangled representations in simulations.} }
Endnote
%0 Conference Paper %T Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA %A Sebastien Lachapelle %A Pau Rodriguez %A Yash Sharma %A Katie E Everett %A Rémi LE PRIOL %A Alexandre Lacoste %A Simon Lacoste-Julien %B Proceedings of the First Conference on Causal Learning and Reasoning %C Proceedings of Machine Learning Research %D 2022 %E Bernhard Schölkopf %E Caroline Uhler %E Kun Zhang %F pmlr-v177-lachapelle22a %I PMLR %P 428--484 %U https://proceedings.mlr.press/v177/lachapelle22a.html %V 177 %X This work introduces a novel principle we call disentanglement via mechanism sparsity regularization, which can be applied when the latent factors of interest depend sparsely on past latent factors and/or observed auxiliary variables. We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors and the sparse causal graphical model that relates them. We develop a rigorous identifiability theory, building on recent nonlinear independent component analysis (ICA) results, that formalizes this principle and shows how the latent variables can be recovered up to permutation if one regularizes the latent mechanisms to be sparse and if some graph connectivity criterion is satisfied by the data generating process. As a special case of our framework, we show how one can leverage unknown-target interventions on the latent factors to disentangle them, thereby drawing further connections between ICA and causality. We propose a VAE-based method in which the latent mechanisms are learned and regularized via binary masks, and validate our theory by showing it learns disentangled representations in simulations.
APA
Lachapelle, S., Rodriguez, P., Sharma, Y., Everett, K.E., PRIOL, R.L., Lacoste, A. & Lacoste-Julien, S.. (2022). Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA. Proceedings of the First Conference on Causal Learning and Reasoning, in Proceedings of Machine Learning Research 177:428-484 Available from https://proceedings.mlr.press/v177/lachapelle22a.html.

Related Material