On the Power of Statistics in Class-Incremental Learning with Pretrained Models

Zhiwen Cao, Yanfeng Li, Shu-Dong Huang, Yalan Ye, Shuyin Xia, Yi Wang, Jiancheng Lv
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:11255-11271, 2026.

Abstract

Recent class-incremental learning (CIL) methods built on large pre-trained vision models have shown that strong performance can be retained even under strict data access constraints. This raises a fundamental question: which properties of pre-trained representations make such recovery possible in the class-incremental setting? In this work, we show that class-level feature statistics play a central role in enabling effective CIL under strong pre-training. When the visual backbone is frozen, maintaining simple class-wise statistical estimators of features can recover a substantial fraction of the performance achieved by static joint training across diverse benchmarks. We make this observation explicit through deliberately minimal reference points built on frozen CLIP representations. In particular, we demonstrate that competitive performance can be achieved without continual parameter updates, by performing class-incremental inference based solely on class-level statistical estimators instantiated from frozen features. Our findings suggest that class-level statistics constitute an important and previously underemphasized component of recent PTM-based CIL approaches, offering a complementary perspective for understanding their strong empirical performance. Our code is available at https://github.com/HdTgon/baseCIL.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-cao26h, title = {On the Power of Statistics in Class-Incremental Learning with Pretrained Models}, author = {Cao, Zhiwen and Li, Yanfeng and Huang, Shu-Dong and Ye, Yalan and Xia, Shuyin and Wang, Yi and Lv, Jiancheng}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {11255--11271}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/cao26h/cao26h.pdf}, url = {https://proceedings.mlr.press/v306/cao26h.html}, abstract = {Recent class-incremental learning (CIL) methods built on large pre-trained vision models have shown that strong performance can be retained even under strict data access constraints. This raises a fundamental question: which properties of pre-trained representations make such recovery possible in the class-incremental setting? In this work, we show that class-level feature statistics play a central role in enabling effective CIL under strong pre-training. When the visual backbone is frozen, maintaining simple class-wise statistical estimators of features can recover a substantial fraction of the performance achieved by static joint training across diverse benchmarks. We make this observation explicit through deliberately minimal reference points built on frozen CLIP representations. In particular, we demonstrate that competitive performance can be achieved without continual parameter updates, by performing class-incremental inference based solely on class-level statistical estimators instantiated from frozen features. Our findings suggest that class-level statistics constitute an important and previously underemphasized component of recent PTM-based CIL approaches, offering a complementary perspective for understanding their strong empirical performance. Our code is available at https://github.com/HdTgon/baseCIL.} }
Endnote
%0 Conference Paper %T On the Power of Statistics in Class-Incremental Learning with Pretrained Models %A Zhiwen Cao %A Yanfeng Li %A Shu-Dong Huang %A Yalan Ye %A Shuyin Xia %A Yi Wang %A Jiancheng Lv %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-cao26h %I PMLR %P 11255--11271 %U https://proceedings.mlr.press/v306/cao26h.html %V 306 %X Recent class-incremental learning (CIL) methods built on large pre-trained vision models have shown that strong performance can be retained even under strict data access constraints. This raises a fundamental question: which properties of pre-trained representations make such recovery possible in the class-incremental setting? In this work, we show that class-level feature statistics play a central role in enabling effective CIL under strong pre-training. When the visual backbone is frozen, maintaining simple class-wise statistical estimators of features can recover a substantial fraction of the performance achieved by static joint training across diverse benchmarks. We make this observation explicit through deliberately minimal reference points built on frozen CLIP representations. In particular, we demonstrate that competitive performance can be achieved without continual parameter updates, by performing class-incremental inference based solely on class-level statistical estimators instantiated from frozen features. Our findings suggest that class-level statistics constitute an important and previously underemphasized component of recent PTM-based CIL approaches, offering a complementary perspective for understanding their strong empirical performance. Our code is available at https://github.com/HdTgon/baseCIL.
APA
Cao, Z., Li, Y., Huang, S., Ye, Y., Xia, S., Wang, Y. & Lv, J.. (2026). On the Power of Statistics in Class-Incremental Learning with Pretrained Models. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:11255-11271 Available from https://proceedings.mlr.press/v306/cao26h.html.

Related Material