Identifiability Conditions for Domain Adaptation

Ishaan Gulrajani, Tatsunori Hashimoto
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:7982-7997, 2022.

Abstract

Domain adaptation algorithms and theory have relied upon an assumption that the observed data uniquely specify the correct correspondence between the domains. Unfortunately, it is unclear under what conditions this identifiability assumption holds, even when restricting ourselves to the case where a correct bijective map between domains exists. We study this bijective domain mapping problem and provide several new sufficient conditions for the identifiability of linear domain maps. As a consequence of our analysis, we show that weak constraints on the third moment tensor suffice for identifiability, prove identifiability for common latent variable models such as topic models, and give a computationally tractable method for generating certificates for the identifiability of linear maps. Inspired by our certification method, we derive a new objective function for domain mapping that explicitly accounts for uncertainty over maps arising from unidentifiability. We demonstrate that our objective leads to improvements in uncertainty quantification and model performance estimation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-gulrajani22a, title = {Identifiability Conditions for Domain Adaptation}, author = {Gulrajani, Ishaan and Hashimoto, Tatsunori}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {7982--7997}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/gulrajani22a/gulrajani22a.pdf}, url = {https://proceedings.mlr.press/v162/gulrajani22a.html}, abstract = {Domain adaptation algorithms and theory have relied upon an assumption that the observed data uniquely specify the correct correspondence between the domains. Unfortunately, it is unclear under what conditions this identifiability assumption holds, even when restricting ourselves to the case where a correct bijective map between domains exists. We study this bijective domain mapping problem and provide several new sufficient conditions for the identifiability of linear domain maps. As a consequence of our analysis, we show that weak constraints on the third moment tensor suffice for identifiability, prove identifiability for common latent variable models such as topic models, and give a computationally tractable method for generating certificates for the identifiability of linear maps. Inspired by our certification method, we derive a new objective function for domain mapping that explicitly accounts for uncertainty over maps arising from unidentifiability. We demonstrate that our objective leads to improvements in uncertainty quantification and model performance estimation.} }
Endnote
%0 Conference Paper %T Identifiability Conditions for Domain Adaptation %A Ishaan Gulrajani %A Tatsunori Hashimoto %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-gulrajani22a %I PMLR %P 7982--7997 %U https://proceedings.mlr.press/v162/gulrajani22a.html %V 162 %X Domain adaptation algorithms and theory have relied upon an assumption that the observed data uniquely specify the correct correspondence between the domains. Unfortunately, it is unclear under what conditions this identifiability assumption holds, even when restricting ourselves to the case where a correct bijective map between domains exists. We study this bijective domain mapping problem and provide several new sufficient conditions for the identifiability of linear domain maps. As a consequence of our analysis, we show that weak constraints on the third moment tensor suffice for identifiability, prove identifiability for common latent variable models such as topic models, and give a computationally tractable method for generating certificates for the identifiability of linear maps. Inspired by our certification method, we derive a new objective function for domain mapping that explicitly accounts for uncertainty over maps arising from unidentifiability. We demonstrate that our objective leads to improvements in uncertainty quantification and model performance estimation.
APA
Gulrajani, I. & Hashimoto, T.. (2022). Identifiability Conditions for Domain Adaptation. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:7982-7997 Available from https://proceedings.mlr.press/v162/gulrajani22a.html.

Related Material