Active Learning with Foundation Model Priors: Efficient Learning under Class Imbalance

Jiancheng Zhang, Meiqing Li, Qi Zhang, Yinglun Zhu
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:156851-156873, 2026.

Abstract

Real-world datasets across image and text domains are often characterized by skewed class distributions and noisy annotations, which jointly degrade model performance, particularly on minority classes. Among existing solutions, active learning offers an effective and efficient paradigm by selectively querying the most informative and balanced samples for annotation. We propose an innovative active learning framework that mitigates class imbalance and selects the most informative samples to annotate. Leveraging foundation model priors, our algorithm enables imbalance-aware co-decisions between foundation model and small model to tackle noisy and imbalanced labels across various domains. We introduce the first study to systematically explore active learning under the dual challenges of label noise and class imbalance across image and text domains. Extensive experiments on imbalanced datasets demonstrate that our method achieves substantial annotation savings—over 50% compared to the best active learning baseline—while preserving performance and robustness to label noise.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-zhang26cz, title = {Active Learning with Foundation Model Priors: Efficient Learning under Class Imbalance}, author = {Zhang, Jiancheng and Li, Meiqing and Zhang, Qi and Zhu, Yinglun}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {156851--156873}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/zhang26cz/zhang26cz.pdf}, url = {https://proceedings.mlr.press/v306/zhang26cz.html}, abstract = {Real-world datasets across image and text domains are often characterized by skewed class distributions and noisy annotations, which jointly degrade model performance, particularly on minority classes. Among existing solutions, active learning offers an effective and efficient paradigm by selectively querying the most informative and balanced samples for annotation. We propose an innovative active learning framework that mitigates class imbalance and selects the most informative samples to annotate. Leveraging foundation model priors, our algorithm enables imbalance-aware co-decisions between foundation model and small model to tackle noisy and imbalanced labels across various domains. We introduce the first study to systematically explore active learning under the dual challenges of label noise and class imbalance across image and text domains. Extensive experiments on imbalanced datasets demonstrate that our method achieves substantial annotation savings—over 50% compared to the best active learning baseline—while preserving performance and robustness to label noise.} }
Endnote
%0 Conference Paper %T Active Learning with Foundation Model Priors: Efficient Learning under Class Imbalance %A Jiancheng Zhang %A Meiqing Li %A Qi Zhang %A Yinglun Zhu %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-zhang26cz %I PMLR %P 156851--156873 %U https://proceedings.mlr.press/v306/zhang26cz.html %V 306 %X Real-world datasets across image and text domains are often characterized by skewed class distributions and noisy annotations, which jointly degrade model performance, particularly on minority classes. Among existing solutions, active learning offers an effective and efficient paradigm by selectively querying the most informative and balanced samples for annotation. We propose an innovative active learning framework that mitigates class imbalance and selects the most informative samples to annotate. Leveraging foundation model priors, our algorithm enables imbalance-aware co-decisions between foundation model and small model to tackle noisy and imbalanced labels across various domains. We introduce the first study to systematically explore active learning under the dual challenges of label noise and class imbalance across image and text domains. Extensive experiments on imbalanced datasets demonstrate that our method achieves substantial annotation savings—over 50% compared to the best active learning baseline—while preserving performance and robustness to label noise.
APA
Zhang, J., Li, M., Zhang, Q. & Zhu, Y.. (2026). Active Learning with Foundation Model Priors: Efficient Learning under Class Imbalance. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:156851-156873 Available from https://proceedings.mlr.press/v306/zhang26cz.html.

Related Material