Identifying Coarse-grained Independent Causal Mechanisms with Self-supervision

Xiaoyang Wang, Klara Nahrstedt, Oluwasanmi O Koyejo
Proceedings of the First Conference on Causal Learning and Reasoning, PMLR 177:877-903, 2022.

Abstract

Among the most effective methods for uncovering high dimensional unstructured data’s generating mechanisms are techniques based on disentangling and learning independent causal mechanisms. However, to identify the disentangled model, previous methods need additional observable variables or do not provide identifiability results. In contrast, this work aims to design an identifiable generative model that approximates the underlying mechanisms from observational data using only self-supervision. Specifically, the generative model uses a degenerate mixture prior to learn mechanisms that generate or transform data. We outline sufficient conditions for an identifiable generative model up to three types of transformations that preserve a coarse-grained disentanglement. Moreover, we propose a self-supervised training method based on these identifiability conditions. We validate our approach on MNIST, FashionMNIST, and Sprites datasets, showing that the proposed method identifies disentangled models – by visualization and evaluating the downstream predictive model’s accuracy under environment shifts.

Cite this Paper

BibTeX
@InProceedings{pmlr-v177-wang22b, title = {Identifying Coarse-grained Independent Causal Mechanisms with Self-supervision}, author = {Wang, Xiaoyang and Nahrstedt, Klara and Koyejo, Oluwasanmi O}, booktitle = {Proceedings of the First Conference on Causal Learning and Reasoning}, pages = {877--903}, year = {2022}, editor = {Schölkopf, Bernhard and Uhler, Caroline and Zhang, Kun}, volume = {177}, series = {Proceedings of Machine Learning Research}, month = {11--13 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v177/wang22b/wang22b.pdf}, url = {https://proceedings.mlr.press/v177/wang22b.html}, abstract = {Among the most effective methods for uncovering high dimensional unstructured data’s generating mechanisms are techniques based on disentangling and learning independent causal mechanisms. However, to identify the disentangled model, previous methods need additional observable variables or do not provide identifiability results. In contrast, this work aims to design an identifiable generative model that approximates the underlying mechanisms from observational data using only self-supervision. Specifically, the generative model uses a degenerate mixture prior to learn mechanisms that generate or transform data. We outline sufficient conditions for an identifiable generative model up to three types of transformations that preserve a coarse-grained disentanglement. Moreover, we propose a self-supervised training method based on these identifiability conditions. We validate our approach on MNIST, FashionMNIST, and Sprites datasets, showing that the proposed method identifies disentangled models – by visualization and evaluating the downstream predictive model’s accuracy under environment shifts.} }
Endnote
%0 Conference Paper %T Identifying Coarse-grained Independent Causal Mechanisms with Self-supervision %A Xiaoyang Wang %A Klara Nahrstedt %A Oluwasanmi O Koyejo %B Proceedings of the First Conference on Causal Learning and Reasoning %C Proceedings of Machine Learning Research %D 2022 %E Bernhard Schölkopf %E Caroline Uhler %E Kun Zhang %F pmlr-v177-wang22b %I PMLR %P 877--903 %U https://proceedings.mlr.press/v177/wang22b.html %V 177 %X Among the most effective methods for uncovering high dimensional unstructured data’s generating mechanisms are techniques based on disentangling and learning independent causal mechanisms. However, to identify the disentangled model, previous methods need additional observable variables or do not provide identifiability results. In contrast, this work aims to design an identifiable generative model that approximates the underlying mechanisms from observational data using only self-supervision. Specifically, the generative model uses a degenerate mixture prior to learn mechanisms that generate or transform data. We outline sufficient conditions for an identifiable generative model up to three types of transformations that preserve a coarse-grained disentanglement. Moreover, we propose a self-supervised training method based on these identifiability conditions. We validate our approach on MNIST, FashionMNIST, and Sprites datasets, showing that the proposed method identifies disentangled models – by visualization and evaluating the downstream predictive model’s accuracy under environment shifts.
APA
Wang, X., Nahrstedt, K. & Koyejo, O.O.. (2022). Identifying Coarse-grained Independent Causal Mechanisms with Self-supervision. Proceedings of the First Conference on Causal Learning and Reasoning, in Proceedings of Machine Learning Research 177:877-903 Available from https://proceedings.mlr.press/v177/wang22b.html.

Related Material