Stochastic Embeddings : A Probabilistic and Geometric Analysis of Out-of-Distribution Behavior

Anthony Nguyen, Emanuel Aldea, Sylvie Le Hégarat-Mascle, Renaud Lustrat
Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence, PMLR 286:3200-3220, 2025.

Abstract

Deep neural networks perform well in many applications but often fail when exposed to out-of-distribution (OoD) inputs. We identify a geometric phenomenon in the embedding space: in-distribution (ID) data show higher variance than OoD data under stochastic perturbations. Using high-dimensional geometry and statistics, we explain this behavior and demonstrate its application in improving OoD detection. Unlike traditional post-hoc methods, our approach integrates uncertainty-aware tools, such as Bayesian approximations, directly into the detection process. Then, we show how considering the unit hypersphere enhances the separation of ID and OoD samples. Our mathematically sound method achieves competitive performance while remaining simple.

Cite this Paper

BibTeX
@InProceedings{pmlr-v286-nguyen25b, title = {Stochastic Embeddings : A Probabilistic and Geometric Analysis of Out-of-Distribution Behavior}, author = {Nguyen, Anthony and Aldea, Emanuel and Le H\'{e}garat-Mascle, Sylvie and Lustrat, Renaud}, booktitle = {Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence}, pages = {3200--3220}, year = {2025}, editor = {Chiappa, Silvia and Magliacane, Sara}, volume = {286}, series = {Proceedings of Machine Learning Research}, month = {21--25 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v286/main/assets/nguyen25b/nguyen25b.pdf}, url = {https://proceedings.mlr.press/v286/nguyen25b.html}, abstract = {Deep neural networks perform well in many applications but often fail when exposed to out-of-distribution (OoD) inputs. We identify a geometric phenomenon in the embedding space: in-distribution (ID) data show higher variance than OoD data under stochastic perturbations. Using high-dimensional geometry and statistics, we explain this behavior and demonstrate its application in improving OoD detection. Unlike traditional post-hoc methods, our approach integrates uncertainty-aware tools, such as Bayesian approximations, directly into the detection process. Then, we show how considering the unit hypersphere enhances the separation of ID and OoD samples. Our mathematically sound method achieves competitive performance while remaining simple.} }
Endnote
%0 Conference Paper %T Stochastic Embeddings : A Probabilistic and Geometric Analysis of Out-of-Distribution Behavior %A Anthony Nguyen %A Emanuel Aldea %A Sylvie Le Hégarat-Mascle %A Renaud Lustrat %B Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2025 %E Silvia Chiappa %E Sara Magliacane %F pmlr-v286-nguyen25b %I PMLR %P 3200--3220 %U https://proceedings.mlr.press/v286/nguyen25b.html %V 286 %X Deep neural networks perform well in many applications but often fail when exposed to out-of-distribution (OoD) inputs. We identify a geometric phenomenon in the embedding space: in-distribution (ID) data show higher variance than OoD data under stochastic perturbations. Using high-dimensional geometry and statistics, we explain this behavior and demonstrate its application in improving OoD detection. Unlike traditional post-hoc methods, our approach integrates uncertainty-aware tools, such as Bayesian approximations, directly into the detection process. Then, we show how considering the unit hypersphere enhances the separation of ID and OoD samples. Our mathematically sound method achieves competitive performance while remaining simple.
APA
Nguyen, A., Aldea, E., Le Hégarat-Mascle, S. & Lustrat, R.. (2025). Stochastic Embeddings : A Probabilistic and Geometric Analysis of Out-of-Distribution Behavior. Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 286:3200-3220 Available from https://proceedings.mlr.press/v286/nguyen25b.html.

Related Material