Rethinking Confidence Scores and Thresholds in Pseudolabeling-based SSL

Harit Vishwakarma, Yi Chen, Satya Sai Srinath Namburi Gnvv, Sui Jiet Tay, Ramya Korlakai Vinayak, Frederic Sala
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:61582-61600, 2025.

Abstract

Modern semi-supervised learning (SSL) methods rely on pseudolabeling and consistency regularization. Pseudolabeling is typically performed by comparing the model’s confidence scores and a predefined threshold. While several heuristics have been proposed to improve threshold selection, the underlying issues of overconfidence and miscalibration in confidence scores remain largely unaddressed, leading to inaccurate pseudolabels, degraded test accuracy, and prolonged training. We take a first-principles approach to learn confidence scores and thresholds with an explicit knob for error. This flexible framework addresses the fundamental question of optimal scores and threshold selection in pseudolabeling. Moreover, it gives practitioners a principled way to control the quality and quantity of pseudolabels. Such control is vital in SSL, where balancing pseudolabel quality and quantity directly affects model performance and training efficiency. Our experiments show that, by integrating this framework with modern SSL methods, we achieve significant improvements in accuracy and training efficiency. In addition, we provide novel insights on the trade-offs between the choices of the error parameter and the end model’s performance.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-vishwakarma25a, title = {Rethinking Confidence Scores and Thresholds in Pseudolabeling-based {SSL}}, author = {Vishwakarma, Harit and Chen, Yi and Namburi Gnvv, Satya Sai Srinath and Tay, Sui Jiet and Vinayak, Ramya Korlakai and Sala, Frederic}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {61582--61600}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/vishwakarma25a/vishwakarma25a.pdf}, url = {https://proceedings.mlr.press/v267/vishwakarma25a.html}, abstract = {Modern semi-supervised learning (SSL) methods rely on pseudolabeling and consistency regularization. Pseudolabeling is typically performed by comparing the model’s confidence scores and a predefined threshold. While several heuristics have been proposed to improve threshold selection, the underlying issues of overconfidence and miscalibration in confidence scores remain largely unaddressed, leading to inaccurate pseudolabels, degraded test accuracy, and prolonged training. We take a first-principles approach to learn confidence scores and thresholds with an explicit knob for error. This flexible framework addresses the fundamental question of optimal scores and threshold selection in pseudolabeling. Moreover, it gives practitioners a principled way to control the quality and quantity of pseudolabels. Such control is vital in SSL, where balancing pseudolabel quality and quantity directly affects model performance and training efficiency. Our experiments show that, by integrating this framework with modern SSL methods, we achieve significant improvements in accuracy and training efficiency. In addition, we provide novel insights on the trade-offs between the choices of the error parameter and the end model’s performance.} }
Endnote
%0 Conference Paper %T Rethinking Confidence Scores and Thresholds in Pseudolabeling-based SSL %A Harit Vishwakarma %A Yi Chen %A Satya Sai Srinath Namburi Gnvv %A Sui Jiet Tay %A Ramya Korlakai Vinayak %A Frederic Sala %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-vishwakarma25a %I PMLR %P 61582--61600 %U https://proceedings.mlr.press/v267/vishwakarma25a.html %V 267 %X Modern semi-supervised learning (SSL) methods rely on pseudolabeling and consistency regularization. Pseudolabeling is typically performed by comparing the model’s confidence scores and a predefined threshold. While several heuristics have been proposed to improve threshold selection, the underlying issues of overconfidence and miscalibration in confidence scores remain largely unaddressed, leading to inaccurate pseudolabels, degraded test accuracy, and prolonged training. We take a first-principles approach to learn confidence scores and thresholds with an explicit knob for error. This flexible framework addresses the fundamental question of optimal scores and threshold selection in pseudolabeling. Moreover, it gives practitioners a principled way to control the quality and quantity of pseudolabels. Such control is vital in SSL, where balancing pseudolabel quality and quantity directly affects model performance and training efficiency. Our experiments show that, by integrating this framework with modern SSL methods, we achieve significant improvements in accuracy and training efficiency. In addition, we provide novel insights on the trade-offs between the choices of the error parameter and the end model’s performance.
APA
Vishwakarma, H., Chen, Y., Namburi Gnvv, S.S.S., Tay, S.J., Vinayak, R.K. & Sala, F.. (2025). Rethinking Confidence Scores and Thresholds in Pseudolabeling-based SSL. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:61582-61600 Available from https://proceedings.mlr.press/v267/vishwakarma25a.html.

Related Material