Neural Collapse by Design: Learning Class Prototypes on the Hypersphere

Panagiotis Koromilas, Theodoros Giannakopoulos, Mihalis Nicolaou, Yannis Panagakis
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:60339-60369, 2026.

Abstract

Supervised classification has a theoretical optimum, Neural Collapse (NC), yet neither of its two dominant paradigms reaches it in practice. Cross entropy (CE) leaves radial degrees of freedom unconstrained and converges to a degenerate geometry, while supervised contrastive learning (SCL) drives features toward NC during pretraining but discards this structure in a post hoc linear probing phase. We show that both paradigms are different appearances of the same method that contrasts prototypes on the unit hypersphere, and that closing the gap requires fixing each at its point of failure. From the CE side, we propose NTCE and NONL, two normalized losses that import contrastive optimization’s missing ingredients into classifier learning: a large effective negative set and decoupled alignment and uniformity terms. From the SCL side, we prove that SCL’s objective already optimizes throughout training for a principled classifier whose weights are the class mean embeddings, making linear probing both redundant and harmful. Empirically, on four benchmarks including ImageNet-1K, NTCE and NONL surpass CE accuracy, closely approximate NC ($\geq 95%$), and match CE’s converged NC on 4/5 metrics in under $7.5%$ of its iterations, while SCL with fixed prototypes matches linear probing without the hours-long classifier training phase. The learned geometry yields $+5.5%$ mean relative improvement in transfer learning, up to $+8.7%$ under severe class imbalance, and improved robustness to corruptions on ImageNet-C. Our work recasts supervised learning as prototype learning on the hypersphere, with NC reached by design.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-koromilas26b, title = {Neural Collapse by Design: Learning Class Prototypes on the Hypersphere}, author = {Koromilas, Panagiotis and Giannakopoulos, Theodoros and Nicolaou, Mihalis and Panagakis, Yannis}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {60339--60369}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/koromilas26b/koromilas26b.pdf}, url = {https://proceedings.mlr.press/v306/koromilas26b.html}, abstract = {Supervised classification has a theoretical optimum, Neural Collapse (NC), yet neither of its two dominant paradigms reaches it in practice. Cross entropy (CE) leaves radial degrees of freedom unconstrained and converges to a degenerate geometry, while supervised contrastive learning (SCL) drives features toward NC during pretraining but discards this structure in a post hoc linear probing phase. We show that both paradigms are different appearances of the same method that contrasts prototypes on the unit hypersphere, and that closing the gap requires fixing each at its point of failure. From the CE side, we propose NTCE and NONL, two normalized losses that import contrastive optimization’s missing ingredients into classifier learning: a large effective negative set and decoupled alignment and uniformity terms. From the SCL side, we prove that SCL’s objective already optimizes throughout training for a principled classifier whose weights are the class mean embeddings, making linear probing both redundant and harmful. Empirically, on four benchmarks including ImageNet-1K, NTCE and NONL surpass CE accuracy, closely approximate NC ($\geq 95%$), and match CE’s converged NC on 4/5 metrics in under $7.5%$ of its iterations, while SCL with fixed prototypes matches linear probing without the hours-long classifier training phase. The learned geometry yields $+5.5%$ mean relative improvement in transfer learning, up to $+8.7%$ under severe class imbalance, and improved robustness to corruptions on ImageNet-C. Our work recasts supervised learning as prototype learning on the hypersphere, with NC reached by design.} }
Endnote
%0 Conference Paper %T Neural Collapse by Design: Learning Class Prototypes on the Hypersphere %A Panagiotis Koromilas %A Theodoros Giannakopoulos %A Mihalis Nicolaou %A Yannis Panagakis %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-koromilas26b %I PMLR %P 60339--60369 %U https://proceedings.mlr.press/v306/koromilas26b.html %V 306 %X Supervised classification has a theoretical optimum, Neural Collapse (NC), yet neither of its two dominant paradigms reaches it in practice. Cross entropy (CE) leaves radial degrees of freedom unconstrained and converges to a degenerate geometry, while supervised contrastive learning (SCL) drives features toward NC during pretraining but discards this structure in a post hoc linear probing phase. We show that both paradigms are different appearances of the same method that contrasts prototypes on the unit hypersphere, and that closing the gap requires fixing each at its point of failure. From the CE side, we propose NTCE and NONL, two normalized losses that import contrastive optimization’s missing ingredients into classifier learning: a large effective negative set and decoupled alignment and uniformity terms. From the SCL side, we prove that SCL’s objective already optimizes throughout training for a principled classifier whose weights are the class mean embeddings, making linear probing both redundant and harmful. Empirically, on four benchmarks including ImageNet-1K, NTCE and NONL surpass CE accuracy, closely approximate NC ($\geq 95%$), and match CE’s converged NC on 4/5 metrics in under $7.5%$ of its iterations, while SCL with fixed prototypes matches linear probing without the hours-long classifier training phase. The learned geometry yields $+5.5%$ mean relative improvement in transfer learning, up to $+8.7%$ under severe class imbalance, and improved robustness to corruptions on ImageNet-C. Our work recasts supervised learning as prototype learning on the hypersphere, with NC reached by design.
APA
Koromilas, P., Giannakopoulos, T., Nicolaou, M. & Panagakis, Y.. (2026). Neural Collapse by Design: Learning Class Prototypes on the Hypersphere. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:60339-60369 Available from https://proceedings.mlr.press/v306/koromilas26b.html.

Related Material