Mechanisms of Introspective Awareness

Uzay Macar, Li Yang, Zifan Wang, Peter Wallich, Emmanuel Ameisen, Jack Lindsey
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:85403-85429, 2026.

Abstract

Recent work shows that LLMs can sometimes detect when steering vectors are injected into their residual stream and identify the injected concept, a phenomenon cited as evidence of "introspective awareness." But what mechanisms underlie this capability, and do they reflect genuine introspective circuitry or more shallow heuristics? We investigate these questions in open-source models and establish three main findings. First, introspection is behaviorally robust: detection achieves moderate true positive rates with 0% false positives across diverse prompts. We also find this capability emerges specifically from post-training rather than pretraining. Second, introspection is not reducible to a single linear confound: anomaly detection relies on distributed MLP computation across multiple directions, implemented by interpretable gate and evidence-carrier features. Third, models possess greater introspective capability than is elicited by default: ablating refusal directions improves detection by 50% and a trained steering vector improves detection by 75%. Overall, our results suggest that introspective awareness is behaviorally robust, grounded in nontrivial internal anomaly detection, and likely could be substantially improved in future models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-macar26a, title = {Mechanisms of Introspective Awareness}, author = {Macar, Uzay and Yang, Li and Wang, Zifan and Wallich, Peter and Ameisen, Emmanuel and Lindsey, Jack}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {85403--85429}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/macar26a/macar26a.pdf}, url = {https://proceedings.mlr.press/v306/macar26a.html}, abstract = {Recent work shows that LLMs can sometimes detect when steering vectors are injected into their residual stream and identify the injected concept, a phenomenon cited as evidence of "introspective awareness." But what mechanisms underlie this capability, and do they reflect genuine introspective circuitry or more shallow heuristics? We investigate these questions in open-source models and establish three main findings. First, introspection is behaviorally robust: detection achieves moderate true positive rates with 0% false positives across diverse prompts. We also find this capability emerges specifically from post-training rather than pretraining. Second, introspection is not reducible to a single linear confound: anomaly detection relies on distributed MLP computation across multiple directions, implemented by interpretable gate and evidence-carrier features. Third, models possess greater introspective capability than is elicited by default: ablating refusal directions improves detection by 50% and a trained steering vector improves detection by 75%. Overall, our results suggest that introspective awareness is behaviorally robust, grounded in nontrivial internal anomaly detection, and likely could be substantially improved in future models.} }
Endnote
%0 Conference Paper %T Mechanisms of Introspective Awareness %A Uzay Macar %A Li Yang %A Zifan Wang %A Peter Wallich %A Emmanuel Ameisen %A Jack Lindsey %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-macar26a %I PMLR %P 85403--85429 %U https://proceedings.mlr.press/v306/macar26a.html %V 306 %X Recent work shows that LLMs can sometimes detect when steering vectors are injected into their residual stream and identify the injected concept, a phenomenon cited as evidence of "introspective awareness." But what mechanisms underlie this capability, and do they reflect genuine introspective circuitry or more shallow heuristics? We investigate these questions in open-source models and establish three main findings. First, introspection is behaviorally robust: detection achieves moderate true positive rates with 0% false positives across diverse prompts. We also find this capability emerges specifically from post-training rather than pretraining. Second, introspection is not reducible to a single linear confound: anomaly detection relies on distributed MLP computation across multiple directions, implemented by interpretable gate and evidence-carrier features. Third, models possess greater introspective capability than is elicited by default: ablating refusal directions improves detection by 50% and a trained steering vector improves detection by 75%. Overall, our results suggest that introspective awareness is behaviorally robust, grounded in nontrivial internal anomaly detection, and likely could be substantially improved in future models.
APA
Macar, U., Yang, L., Wang, Z., Wallich, P., Ameisen, E. & Lindsey, J.. (2026). Mechanisms of Introspective Awareness. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:85403-85429 Available from https://proceedings.mlr.press/v306/macar26a.html.

Related Material