Towards Cross-Modal Causal Structure and Representation Learning

Haiyi Mao, Hongfu Liu, Jason Xiaotian Dou, Panayiotis V. Benos
Proceedings of the 2nd Machine Learning for Health symposium, PMLR 193:120-140, 2022.

Abstract

Does the SARS-CoV-2 virus cause patients’ chest X-Rays ground-glass opacities? Does an IDH-mutation cause differences in patients’ MRI images? Conventional causal discovery algorithms, although well developed to uncover the cause-effect relationships on structured data, cannot elucidate causal relations between unstructured images and structured scalar variables due to the complexity of the former. In this paper, we consider causal discovery between images and structured (scalar) variables. Specifically, we derive low dimensional image representations to analyze with structured variables. We propose a two-module amortized variational algorithm named Cross-Modal Variational Causal representation and structure Learning (CMCL). CMCL jointly learns identifiable representations given a set of independent structured variables and causal relations via formulating latent representations and structured variables into a direct acyclic graph. Moreover, we further enforce counterfactual invariance/variance onto representations. We demonstrate that CMCL outperforms other related methods on synthetic datasets and validate causal relations on semi-synthetic datasets by visualization.

Cite this Paper

BibTeX
@InProceedings{pmlr-v193-mao22a, title = {Towards Cross-Modal Causal Structure and Representation Learning}, author = {Mao, Haiyi and Liu, Hongfu and Dou, Jason Xiaotian and Benos, Panayiotis V.}, booktitle = {Proceedings of the 2nd Machine Learning for Health symposium}, pages = {120--140}, year = {2022}, editor = {Parziale, Antonio and Agrawal, Monica and Joshi, Shalmali and Chen, Irene Y. and Tang, Shengpu and Oala, Luis and Subbaswamy, Adarsh}, volume = {193}, series = {Proceedings of Machine Learning Research}, month = {28 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v193/mao22a/mao22a.pdf}, url = {https://proceedings.mlr.press/v193/mao22a.html}, abstract = {Does the SARS-CoV-2 virus cause patients’ chest X-Rays ground-glass opacities? Does an IDH-mutation cause differences in patients’ MRI images? Conventional causal discovery algorithms, although well developed to uncover the cause-effect relationships on structured data, cannot elucidate causal relations between unstructured images and structured scalar variables due to the complexity of the former. In this paper, we consider causal discovery between images and structured (scalar) variables. Specifically, we derive low dimensional image representations to analyze with structured variables. We propose a two-module amortized variational algorithm named Cross-Modal Variational Causal representation and structure Learning (CMCL). CMCL jointly learns identifiable representations given a set of independent structured variables and causal relations via formulating latent representations and structured variables into a direct acyclic graph. Moreover, we further enforce counterfactual invariance/variance onto representations. We demonstrate that CMCL outperforms other related methods on synthetic datasets and validate causal relations on semi-synthetic datasets by visualization.} }
Endnote
%0 Conference Paper %T Towards Cross-Modal Causal Structure and Representation Learning %A Haiyi Mao %A Hongfu Liu %A Jason Xiaotian Dou %A Panayiotis V. Benos %B Proceedings of the 2nd Machine Learning for Health symposium %C Proceedings of Machine Learning Research %D 2022 %E Antonio Parziale %E Monica Agrawal %E Shalmali Joshi %E Irene Y. Chen %E Shengpu Tang %E Luis Oala %E Adarsh Subbaswamy %F pmlr-v193-mao22a %I PMLR %P 120--140 %U https://proceedings.mlr.press/v193/mao22a.html %V 193 %X Does the SARS-CoV-2 virus cause patients’ chest X-Rays ground-glass opacities? Does an IDH-mutation cause differences in patients’ MRI images? Conventional causal discovery algorithms, although well developed to uncover the cause-effect relationships on structured data, cannot elucidate causal relations between unstructured images and structured scalar variables due to the complexity of the former. In this paper, we consider causal discovery between images and structured (scalar) variables. Specifically, we derive low dimensional image representations to analyze with structured variables. We propose a two-module amortized variational algorithm named Cross-Modal Variational Causal representation and structure Learning (CMCL). CMCL jointly learns identifiable representations given a set of independent structured variables and causal relations via formulating latent representations and structured variables into a direct acyclic graph. Moreover, we further enforce counterfactual invariance/variance onto representations. We demonstrate that CMCL outperforms other related methods on synthetic datasets and validate causal relations on semi-synthetic datasets by visualization.
APA
Mao, H., Liu, H., Dou, J.X. & Benos, P.V.. (2022). Towards Cross-Modal Causal Structure and Representation Learning. Proceedings of the 2nd Machine Learning for Health symposium, in Proceedings of Machine Learning Research 193:120-140 Available from https://proceedings.mlr.press/v193/mao22a.html.

Related Material