Tilt and Average : Geometric Adjustment of the Last Layer for Recalibration

Gyusang Cho, Chan-Hyun Youn
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:8609-8628, 2024.

Abstract

After the revelation that neural networks tend to produce overconfident predictions, the problem of calibration, which aims to align confidence with accuracy to enhance the reliability of predictions, has gained significant importance. Several solutions based on calibration maps have been proposed to address the problem of recalibrating a trained classifier using additional datasets. In this paper, we offer an algorithm that transforms the weights of the last layer of the classifier, distinct from the calibration-map-based approach. We concentrate on the geometry of the final linear layer, specifically its angular aspect, and adjust the weights of the corresponding layer. We name the method Tilt and Average, and validate the calibration effect empirically and theoretically. Through this, we demonstrate that our approach, in addition to the existing calibration-map-based techniques, can yield improved calibration performance.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-cho24g, title = {Tilt and Average : Geometric Adjustment of the Last Layer for Recalibration}, author = {Cho, Gyusang and Youn, Chan-Hyun}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {8609--8628}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/cho24g/cho24g.pdf}, url = {https://proceedings.mlr.press/v235/cho24g.html}, abstract = {After the revelation that neural networks tend to produce overconfident predictions, the problem of calibration, which aims to align confidence with accuracy to enhance the reliability of predictions, has gained significant importance. Several solutions based on calibration maps have been proposed to address the problem of recalibrating a trained classifier using additional datasets. In this paper, we offer an algorithm that transforms the weights of the last layer of the classifier, distinct from the calibration-map-based approach. We concentrate on the geometry of the final linear layer, specifically its angular aspect, and adjust the weights of the corresponding layer. We name the method Tilt and Average, and validate the calibration effect empirically and theoretically. Through this, we demonstrate that our approach, in addition to the existing calibration-map-based techniques, can yield improved calibration performance.} }
Endnote
%0 Conference Paper %T Tilt and Average : Geometric Adjustment of the Last Layer for Recalibration %A Gyusang Cho %A Chan-Hyun Youn %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-cho24g %I PMLR %P 8609--8628 %U https://proceedings.mlr.press/v235/cho24g.html %V 235 %X After the revelation that neural networks tend to produce overconfident predictions, the problem of calibration, which aims to align confidence with accuracy to enhance the reliability of predictions, has gained significant importance. Several solutions based on calibration maps have been proposed to address the problem of recalibrating a trained classifier using additional datasets. In this paper, we offer an algorithm that transforms the weights of the last layer of the classifier, distinct from the calibration-map-based approach. We concentrate on the geometry of the final linear layer, specifically its angular aspect, and adjust the weights of the corresponding layer. We name the method Tilt and Average, and validate the calibration effect empirically and theoretically. Through this, we demonstrate that our approach, in addition to the existing calibration-map-based techniques, can yield improved calibration performance.
APA
Cho, G. & Youn, C.. (2024). Tilt and Average : Geometric Adjustment of the Last Layer for Recalibration. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:8609-8628 Available from https://proceedings.mlr.press/v235/cho24g.html.

Related Material