A Discriminative Technique for Multiple-Source Adaptation

Corinna Cortes, Mehryar Mohri, Ananda Theertha Suresh, Ningshan Zhang
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:2132-2143, 2021.

Abstract

We present a new discriminative technique for the multiple-source adaptation (MSA) problem. Unlike previous work, which relies on density estimation for each source domain, our solution only requires conditional probabilities that can be straightforwardly accurately estimated from unlabeled data from the source domains. We give a detailed analysis of our new technique, including general guarantees based on Rényi divergences, and learning bounds when conditional Maxent is used for estimating conditional probabilities for a point to belong to a source domain. We show that these guarantees compare favorably to those that can be derived for the generative solution, using kernel density estimation. Our experiments with real-world applications further demonstrate that our new discriminative MSA algorithm outperforms the previous generative solution as well as other domain adaptation baselines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-cortes21b, title = {A Discriminative Technique for Multiple-Source Adaptation}, author = {Cortes, Corinna and Mohri, Mehryar and Suresh, Ananda Theertha and Zhang, Ningshan}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {2132--2143}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/cortes21b/cortes21b.pdf}, url = {https://proceedings.mlr.press/v139/cortes21b.html}, abstract = {We present a new discriminative technique for the multiple-source adaptation (MSA) problem. Unlike previous work, which relies on density estimation for each source domain, our solution only requires conditional probabilities that can be straightforwardly accurately estimated from unlabeled data from the source domains. We give a detailed analysis of our new technique, including general guarantees based on Rényi divergences, and learning bounds when conditional Maxent is used for estimating conditional probabilities for a point to belong to a source domain. We show that these guarantees compare favorably to those that can be derived for the generative solution, using kernel density estimation. Our experiments with real-world applications further demonstrate that our new discriminative MSA algorithm outperforms the previous generative solution as well as other domain adaptation baselines.} }
Endnote
%0 Conference Paper %T A Discriminative Technique for Multiple-Source Adaptation %A Corinna Cortes %A Mehryar Mohri %A Ananda Theertha Suresh %A Ningshan Zhang %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-cortes21b %I PMLR %P 2132--2143 %U https://proceedings.mlr.press/v139/cortes21b.html %V 139 %X We present a new discriminative technique for the multiple-source adaptation (MSA) problem. Unlike previous work, which relies on density estimation for each source domain, our solution only requires conditional probabilities that can be straightforwardly accurately estimated from unlabeled data from the source domains. We give a detailed analysis of our new technique, including general guarantees based on Rényi divergences, and learning bounds when conditional Maxent is used for estimating conditional probabilities for a point to belong to a source domain. We show that these guarantees compare favorably to those that can be derived for the generative solution, using kernel density estimation. Our experiments with real-world applications further demonstrate that our new discriminative MSA algorithm outperforms the previous generative solution as well as other domain adaptation baselines.
APA
Cortes, C., Mohri, M., Suresh, A.T. & Zhang, N.. (2021). A Discriminative Technique for Multiple-Source Adaptation. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:2132-2143 Available from https://proceedings.mlr.press/v139/cortes21b.html.

Related Material