Bridging Theory and Algorithm for Domain Adaptation

Yuchen Zhang, Tianle Liu, Mingsheng Long, Michael Jordan
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:7404-7413, 2019.

Abstract

This paper addresses the problem of unsupervised domain adaption from theoretical and algorithmic perspectives. Existing domain adaptation theories naturally imply minimax optimization algorithms, which connect well with the domain adaptation methods based on adversarial learning. However, several disconnections still exist and form the gap between theory and algorithm. We extend previous theories (Mansour et al., 2009c; Ben-David et al., 2010) to multiclass classification in domain adaptation, where classifiers based on the scoring functions and margin loss are standard choices in algorithm design. We introduce Margin Disparity Discrepancy, a novel measurement with rigorous generalization bounds, tailored to the distribution comparison with the asymmetric margin loss, and to the minimax optimization for easier training. Our theory can be seamlessly transformed into an adversarial learning algorithm for domain adaptation, successfully bridging the gap between theory and algorithm. A series of empirical studies show that our algorithm achieves the state of the art accuracies on challenging domain adaptation tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-zhang19i, title = {Bridging Theory and Algorithm for Domain Adaptation}, author = {Zhang, Yuchen and Liu, Tianle and Long, Mingsheng and Jordan, Michael}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {7404--7413}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/zhang19i/zhang19i.pdf}, url = {https://proceedings.mlr.press/v97/zhang19i.html}, abstract = {This paper addresses the problem of unsupervised domain adaption from theoretical and algorithmic perspectives. Existing domain adaptation theories naturally imply minimax optimization algorithms, which connect well with the domain adaptation methods based on adversarial learning. However, several disconnections still exist and form the gap between theory and algorithm. We extend previous theories (Mansour et al., 2009c; Ben-David et al., 2010) to multiclass classification in domain adaptation, where classifiers based on the scoring functions and margin loss are standard choices in algorithm design. We introduce Margin Disparity Discrepancy, a novel measurement with rigorous generalization bounds, tailored to the distribution comparison with the asymmetric margin loss, and to the minimax optimization for easier training. Our theory can be seamlessly transformed into an adversarial learning algorithm for domain adaptation, successfully bridging the gap between theory and algorithm. A series of empirical studies show that our algorithm achieves the state of the art accuracies on challenging domain adaptation tasks.} }
Endnote
%0 Conference Paper %T Bridging Theory and Algorithm for Domain Adaptation %A Yuchen Zhang %A Tianle Liu %A Mingsheng Long %A Michael Jordan %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-zhang19i %I PMLR %P 7404--7413 %U https://proceedings.mlr.press/v97/zhang19i.html %V 97 %X This paper addresses the problem of unsupervised domain adaption from theoretical and algorithmic perspectives. Existing domain adaptation theories naturally imply minimax optimization algorithms, which connect well with the domain adaptation methods based on adversarial learning. However, several disconnections still exist and form the gap between theory and algorithm. We extend previous theories (Mansour et al., 2009c; Ben-David et al., 2010) to multiclass classification in domain adaptation, where classifiers based on the scoring functions and margin loss are standard choices in algorithm design. We introduce Margin Disparity Discrepancy, a novel measurement with rigorous generalization bounds, tailored to the distribution comparison with the asymmetric margin loss, and to the minimax optimization for easier training. Our theory can be seamlessly transformed into an adversarial learning algorithm for domain adaptation, successfully bridging the gap between theory and algorithm. A series of empirical studies show that our algorithm achieves the state of the art accuracies on challenging domain adaptation tasks.
APA
Zhang, Y., Liu, T., Long, M. & Jordan, M.. (2019). Bridging Theory and Algorithm for Domain Adaptation. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:7404-7413 Available from https://proceedings.mlr.press/v97/zhang19i.html.

Related Material