Detecting Out of Distribution Samples using Class Centered Residual Energy in the Discarded PCA Subspace with Weight Alignment

Shreen Gul, Mohamed Elmahallawy, Ardhendu Tripathy, Sanjay Madria
Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337:1790-1803, 2026.

Abstract

Principal component analysis ({PCA}) has recently been adopted for out-of-distribution ({OOD}) detection, yet most existing methods focus on dominant, high-variance directions of the feature space. This energy-retaining perspective overlooks a crucial fact: discriminative signals separating in distribution (ID) and {OOD} samples often lie in low variance residual components that are typically discarded. Furthermore, prior {PCA}-based scores rely solely on feature geometry, limiting robustness under distributional shift. In this work, we revisit {PCA} for {OOD} detection from an uncertainty aware perspective. Specifically, we propose a residual-aware {OOD} detection framework that explicitly models the discarded subspace of a shared within-class {PCA} basis. By constructing a residual variation score, we capture deviations that are ignored by the leading principal components. To further enhance reliability, we introduce a complementary classifier-weight alignment score that measures the consistency between the selected class-centered residual and the corresponding classifier weight vector. This dual-signal design enables reliable detection even when {OOD} samples exhibit high energy in the dominant subspaces. Extensive experiments across convolutional and transformer backbones show consistent improvements over strong baselines in AUROC and FPR at 95% TPR. This shows that modeling residual structure and integrating geometric and classifier-aware signals leads to more principled and robust {OOD} detection. The code to reproduce the results is available at https://github.com/sgchr273/CREWA.git

Cite this Paper

BibTeX
@InProceedings{pmlr-v337-gul26a, title = {Detecting Out of Distribution Samples using Class Centered Residual Energy in the Discarded {PCA} Subspace with Weight Alignment}, author = {Gul, Shreen and Elmahallawy, Mohamed and Tripathy, Ardhendu and Madria, Sanjay}, booktitle = {Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence}, pages = {1790--1803}, year = {2026}, editor = {Perković, Emilija and Malinsky, Daniel}, volume = {337}, series = {Proceedings of Machine Learning Research}, month = {17--21 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v337/main/assets/gul26a/gul26a.pdf}, url = {https://proceedings.mlr.press/v337/gul26a.html}, abstract = {Principal component analysis ({PCA}) has recently been adopted for out-of-distribution ({OOD}) detection, yet most existing methods focus on dominant, high-variance directions of the feature space. This energy-retaining perspective overlooks a crucial fact: discriminative signals separating in distribution (ID) and {OOD} samples often lie in low variance residual components that are typically discarded. Furthermore, prior {PCA}-based scores rely solely on feature geometry, limiting robustness under distributional shift. In this work, we revisit {PCA} for {OOD} detection from an uncertainty aware perspective. Specifically, we propose a residual-aware {OOD} detection framework that explicitly models the discarded subspace of a shared within-class {PCA} basis. By constructing a residual variation score, we capture deviations that are ignored by the leading principal components. To further enhance reliability, we introduce a complementary classifier-weight alignment score that measures the consistency between the selected class-centered residual and the corresponding classifier weight vector. This dual-signal design enables reliable detection even when {OOD} samples exhibit high energy in the dominant subspaces. Extensive experiments across convolutional and transformer backbones show consistent improvements over strong baselines in AUROC and FPR at 95% TPR. This shows that modeling residual structure and integrating geometric and classifier-aware signals leads to more principled and robust {OOD} detection. The code to reproduce the results is available at https://github.com/sgchr273/CREWA.git} }
Endnote
%0 Conference Paper %T Detecting Out of Distribution Samples using Class Centered Residual Energy in the Discarded PCA Subspace with Weight Alignment %A Shreen Gul %A Mohamed Elmahallawy %A Ardhendu Tripathy %A Sanjay Madria %B Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Emilija Perković %E Daniel Malinsky %F pmlr-v337-gul26a %I PMLR %P 1790--1803 %U https://proceedings.mlr.press/v337/gul26a.html %V 337 %X Principal component analysis ({PCA}) has recently been adopted for out-of-distribution ({OOD}) detection, yet most existing methods focus on dominant, high-variance directions of the feature space. This energy-retaining perspective overlooks a crucial fact: discriminative signals separating in distribution (ID) and {OOD} samples often lie in low variance residual components that are typically discarded. Furthermore, prior {PCA}-based scores rely solely on feature geometry, limiting robustness under distributional shift. In this work, we revisit {PCA} for {OOD} detection from an uncertainty aware perspective. Specifically, we propose a residual-aware {OOD} detection framework that explicitly models the discarded subspace of a shared within-class {PCA} basis. By constructing a residual variation score, we capture deviations that are ignored by the leading principal components. To further enhance reliability, we introduce a complementary classifier-weight alignment score that measures the consistency between the selected class-centered residual and the corresponding classifier weight vector. This dual-signal design enables reliable detection even when {OOD} samples exhibit high energy in the dominant subspaces. Extensive experiments across convolutional and transformer backbones show consistent improvements over strong baselines in AUROC and FPR at 95% TPR. This shows that modeling residual structure and integrating geometric and classifier-aware signals leads to more principled and robust {OOD} detection. The code to reproduce the results is available at https://github.com/sgchr273/CREWA.git
APA
Gul, S., Elmahallawy, M., Tripathy, A. & Madria, S.. (2026). Detecting Out of Distribution Samples using Class Centered Residual Energy in the Discarded PCA Subspace with Weight Alignment. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 337:1790-1803 Available from https://proceedings.mlr.press/v337/gul26a.html.

Related Material