Bidirectional Adaptation for Robust Semi-Supervised Learning with Inconsistent Data Distributions

Lin-Han Jia, Lan-Zhe Guo, Zhi Zhou, Jie-Jing Shao, Yuke Xiang, Yu-Feng Li
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:14886-14901, 2023.

Abstract

Semi-supervised learning (SSL) suffers from severe performance degradation when labeled and unlabeled data come from inconsistent data distributions. However, there is still a lack of sufficient theoretical guidance on how to alleviate this problem. In this paper, we propose a general theoretical framework that demonstrates how distribution discrepancies caused by pseudo-label predictions and target predictions can lead to severe generalization errors. Through theoretical analysis, we identify three main reasons why previous SSL algorithms cannot perform well with inconsistent distributions: coupling between the pseudo-label predictor and the target predictor, biased pseudo labels, and restricted sample weights. To address these challenges, we introduce a practical framework called Bidirectional Adaptation that can adapt to the distribution of unlabeled data for debiased pseudo-label prediction and to the target distribution for debiased target prediction, thereby mitigating these shortcomings. Extensive experimental results demonstrate the effectiveness of our proposed framework.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-jia23a, title = {Bidirectional Adaptation for Robust Semi-Supervised Learning with Inconsistent Data Distributions}, author = {Jia, Lin-Han and Guo, Lan-Zhe and Zhou, Zhi and Shao, Jie-Jing and Xiang, Yuke and Li, Yu-Feng}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {14886--14901}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/jia23a/jia23a.pdf}, url = {https://proceedings.mlr.press/v202/jia23a.html}, abstract = {Semi-supervised learning (SSL) suffers from severe performance degradation when labeled and unlabeled data come from inconsistent data distributions. However, there is still a lack of sufficient theoretical guidance on how to alleviate this problem. In this paper, we propose a general theoretical framework that demonstrates how distribution discrepancies caused by pseudo-label predictions and target predictions can lead to severe generalization errors. Through theoretical analysis, we identify three main reasons why previous SSL algorithms cannot perform well with inconsistent distributions: coupling between the pseudo-label predictor and the target predictor, biased pseudo labels, and restricted sample weights. To address these challenges, we introduce a practical framework called Bidirectional Adaptation that can adapt to the distribution of unlabeled data for debiased pseudo-label prediction and to the target distribution for debiased target prediction, thereby mitigating these shortcomings. Extensive experimental results demonstrate the effectiveness of our proposed framework.} }
Endnote
%0 Conference Paper %T Bidirectional Adaptation for Robust Semi-Supervised Learning with Inconsistent Data Distributions %A Lin-Han Jia %A Lan-Zhe Guo %A Zhi Zhou %A Jie-Jing Shao %A Yuke Xiang %A Yu-Feng Li %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-jia23a %I PMLR %P 14886--14901 %U https://proceedings.mlr.press/v202/jia23a.html %V 202 %X Semi-supervised learning (SSL) suffers from severe performance degradation when labeled and unlabeled data come from inconsistent data distributions. However, there is still a lack of sufficient theoretical guidance on how to alleviate this problem. In this paper, we propose a general theoretical framework that demonstrates how distribution discrepancies caused by pseudo-label predictions and target predictions can lead to severe generalization errors. Through theoretical analysis, we identify three main reasons why previous SSL algorithms cannot perform well with inconsistent distributions: coupling between the pseudo-label predictor and the target predictor, biased pseudo labels, and restricted sample weights. To address these challenges, we introduce a practical framework called Bidirectional Adaptation that can adapt to the distribution of unlabeled data for debiased pseudo-label prediction and to the target distribution for debiased target prediction, thereby mitigating these shortcomings. Extensive experimental results demonstrate the effectiveness of our proposed framework.
APA
Jia, L., Guo, L., Zhou, Z., Shao, J., Xiang, Y. & Li, Y.. (2023). Bidirectional Adaptation for Robust Semi-Supervised Learning with Inconsistent Data Distributions. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:14886-14901 Available from https://proceedings.mlr.press/v202/jia23a.html.

Related Material