The Incomplete Rosetta Stone problem: Identifiability results for Multi-view Nonlinear ICA

Luigi Gresele, Paul K. Rubenstein, Arash Mehrjou, Francesco Locatello, Bernhard Schölkopf
Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, PMLR 115:217-227, 2020.

Abstract

We consider the problem of recovering a common latent source with independent components from multiple views. This applies to settings in which a variable is measured with multiple experimental modalities, and where the goal is to synthesize the disparate measurements into a single unified representation. We consider the case that the observed views are a nonlinear mixing of component-wise corruptions of the sources. When the views are considered separately, this reduces to nonlinear Independent Component Analysis (ICA) for which it is provably impossible to undo the mixing. We present novel identifiability proofs that this is possible when the multiple views are considered jointly, showing that the mixing can theoretically be undone using function approximators such as deep neural networks. In contrast to known identifiability results for nonlinear ICA, we prove that independent latent sources with arbitrary mixing can be recovered as long as multiple, sufficiently different noisy views are available.

Cite this Paper

BibTeX
@InProceedings{pmlr-v115-gresele20a, title = {The Incomplete Rosetta Stone problem: Identifiability results for Multi-view Nonlinear ICA}, author = {Gresele, Luigi and Rubenstein, Paul K. and Mehrjou, Arash and Locatello, Francesco and Sch{\"{o}}lkopf, Bernhard}, booktitle = {Proceedings of The 35th Uncertainty in Artificial Intelligence Conference}, pages = {217--227}, year = {2020}, editor = {Adams, Ryan P. and Gogate, Vibhav}, volume = {115}, series = {Proceedings of Machine Learning Research}, month = {22--25 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v115/gresele20a/gresele20a.pdf}, url = {https://proceedings.mlr.press/v115/gresele20a.html}, abstract = {We consider the problem of recovering a common latent source with independent components from multiple views. This applies to settings in which a variable is measured with multiple experimental modalities, and where the goal is to synthesize the disparate measurements into a single unified representation. We consider the case that the observed views are a nonlinear mixing of component-wise corruptions of the sources. When the views are considered separately, this reduces to nonlinear Independent Component Analysis (ICA) for which it is provably impossible to undo the mixing. We present novel identifiability proofs that this is possible when the multiple views are considered jointly, showing that the mixing can theoretically be undone using function approximators such as deep neural networks. In contrast to known identifiability results for nonlinear ICA, we prove that independent latent sources with arbitrary mixing can be recovered as long as multiple, sufficiently different noisy views are available. } }
Endnote
%0 Conference Paper %T The Incomplete Rosetta Stone problem: Identifiability results for Multi-view Nonlinear ICA %A Luigi Gresele %A Paul K. Rubenstein %A Arash Mehrjou %A Francesco Locatello %A Bernhard Schölkopf %B Proceedings of The 35th Uncertainty in Artificial Intelligence Conference %C Proceedings of Machine Learning Research %D 2020 %E Ryan P. Adams %E Vibhav Gogate %F pmlr-v115-gresele20a %I PMLR %P 217--227 %U https://proceedings.mlr.press/v115/gresele20a.html %V 115 %X We consider the problem of recovering a common latent source with independent components from multiple views. This applies to settings in which a variable is measured with multiple experimental modalities, and where the goal is to synthesize the disparate measurements into a single unified representation. We consider the case that the observed views are a nonlinear mixing of component-wise corruptions of the sources. When the views are considered separately, this reduces to nonlinear Independent Component Analysis (ICA) for which it is provably impossible to undo the mixing. We present novel identifiability proofs that this is possible when the multiple views are considered jointly, showing that the mixing can theoretically be undone using function approximators such as deep neural networks. In contrast to known identifiability results for nonlinear ICA, we prove that independent latent sources with arbitrary mixing can be recovered as long as multiple, sufficiently different noisy views are available.
APA
Gresele, L., Rubenstein, P.K., Mehrjou, A., Locatello, F. & Schölkopf, B.. (2020). The Incomplete Rosetta Stone problem: Identifiability results for Multi-view Nonlinear ICA. Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, in Proceedings of Machine Learning Research 115:217-227 Available from https://proceedings.mlr.press/v115/gresele20a.html.

Related Material