Great Models Think Alike: Improving Model Reliability via Inter-Model Latent Agreement

Ailin Deng, Miao Xiong, Bryan Hooi
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:7675-7693, 2023.

Abstract

Reliable application of machine learning is of primary importance to the practical deployment of deep learning methods. A fundamental challenge is that models are often unreliable due to overconfidence. In this paper, we estimate a model’s reliability by measuring the agreement between its latent space, and the latent space of a foundation model. However, it is challenging to measure the agreement between two different latent spaces due to their incoherence, e.g., arbitrary rotations and different dimensionality. To overcome this incoherence issue, we design a neighborhood agreement measure between latent spaces and find that this agreement is surprisingly well-correlated with the reliability of a model’s predictions. Further, we show that fusing neighborhood agreement into a model’s predictive confidence in a post-hoc way significantly improves its reliability. Theoretical analysis and extensive experiments on failure detection across various datasets verify the effectiveness of our method on both in-distribution and out-of-distribution settings.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-deng23f, title = {Great Models Think Alike: Improving Model Reliability via Inter-Model Latent Agreement}, author = {Deng, Ailin and Xiong, Miao and Hooi, Bryan}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {7675--7693}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/deng23f/deng23f.pdf}, url = {https://proceedings.mlr.press/v202/deng23f.html}, abstract = {Reliable application of machine learning is of primary importance to the practical deployment of deep learning methods. A fundamental challenge is that models are often unreliable due to overconfidence. In this paper, we estimate a model’s reliability by measuring the agreement between its latent space, and the latent space of a foundation model. However, it is challenging to measure the agreement between two different latent spaces due to their incoherence, e.g., arbitrary rotations and different dimensionality. To overcome this incoherence issue, we design a neighborhood agreement measure between latent spaces and find that this agreement is surprisingly well-correlated with the reliability of a model’s predictions. Further, we show that fusing neighborhood agreement into a model’s predictive confidence in a post-hoc way significantly improves its reliability. Theoretical analysis and extensive experiments on failure detection across various datasets verify the effectiveness of our method on both in-distribution and out-of-distribution settings.} }
Endnote
%0 Conference Paper %T Great Models Think Alike: Improving Model Reliability via Inter-Model Latent Agreement %A Ailin Deng %A Miao Xiong %A Bryan Hooi %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-deng23f %I PMLR %P 7675--7693 %U https://proceedings.mlr.press/v202/deng23f.html %V 202 %X Reliable application of machine learning is of primary importance to the practical deployment of deep learning methods. A fundamental challenge is that models are often unreliable due to overconfidence. In this paper, we estimate a model’s reliability by measuring the agreement between its latent space, and the latent space of a foundation model. However, it is challenging to measure the agreement between two different latent spaces due to their incoherence, e.g., arbitrary rotations and different dimensionality. To overcome this incoherence issue, we design a neighborhood agreement measure between latent spaces and find that this agreement is surprisingly well-correlated with the reliability of a model’s predictions. Further, we show that fusing neighborhood agreement into a model’s predictive confidence in a post-hoc way significantly improves its reliability. Theoretical analysis and extensive experiments on failure detection across various datasets verify the effectiveness of our method on both in-distribution and out-of-distribution settings.
APA
Deng, A., Xiong, M. & Hooi, B.. (2023). Great Models Think Alike: Improving Model Reliability via Inter-Model Latent Agreement. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:7675-7693 Available from https://proceedings.mlr.press/v202/deng23f.html.

Related Material